Nov 04 03:16:23.789340 ip-10-0-128-50 systemd[1]: Starting Kubernetes Kubelet...
Nov 04 03:16:24.221036 ip-10-0-128-50 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 03:16:24.221036 ip-10-0-128-50 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 04 03:16:24.221036 ip-10-0-128-50 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 03:16:24.221036 ip-10-0-128-50 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 03:16:24.225600 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225514    2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:16:24.225600 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225599    2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:16:24.225600 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225603    2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:16:24.225693 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225607    2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:16:24.225693 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225610    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:16:24.225693 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225612    2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:16:24.225693 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225615    2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:16:24.225693 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225618    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:16:24.225693 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225620    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:16:24.225693 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225623    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:16:24.225693 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225626    2579 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:16:24.225693 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225628    2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:16:24.225693 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225631    2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:16:24.225693 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225634    2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:16:24.225693 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225636    2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:16:24.225693 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225639    2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:16:24.225693 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225641    2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:16:24.225693 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225644    2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:16:24.225693 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225648    2579 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:16:24.225693 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225650    2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:16:24.225693 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225658    2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:16:24.225693 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225661    2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:16:24.226192 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225663    2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:16:24.226192 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225666    2579 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:16:24.226192 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225668    2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:16:24.226192 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225671    2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:16:24.226192 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225674    2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:16:24.226192 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225676    2579 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:16:24.226192 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225679    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:16:24.226192 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225681    2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:16:24.226192 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225684    2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:16:24.226192 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225687    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:16:24.226192 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225689    2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:16:24.226192 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225692    2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:16:24.226192 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225694    2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:16:24.226192 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225697    2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:16:24.226192 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225700    2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:16:24.226192 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225703    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:16:24.226192 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225706    2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:16:24.226192 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225709    2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:16:24.226192 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225711    2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:16:24.226192 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225714    2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:16:24.226657 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225717    2579 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:16:24.226657 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225719    2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:16:24.226657 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225722    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:16:24.226657 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225724    2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:16:24.226657 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225727    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:16:24.226657 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225729    2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:16:24.226657 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225732    2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:16:24.226657 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225735    2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:16:24.226657 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225737    2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:16:24.226657 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225740    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:16:24.226657 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225742    2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:16:24.226657 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225745    2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:16:24.226657 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225747    2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:16:24.226657 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225751    2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:16:24.226657 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225753    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:16:24.226657 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225757    2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:16:24.226657 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225762    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:16:24.226657 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225766    2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:16:24.226657 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225769    2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:16:24.227114 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225772    2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:16:24.227114 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225774    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:16:24.227114 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225777    2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:16:24.227114 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225780    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:16:24.227114 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225783    2579 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:16:24.227114 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225785    2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:16:24.227114 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225788    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:16:24.227114 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225790    2579 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:16:24.227114 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225793    2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:16:24.227114 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225796    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:16:24.227114 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225799    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:16:24.227114 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225801    2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:16:24.227114 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225804    2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:16:24.227114 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225807    2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:16:24.227114 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225809    2579 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:16:24.227114 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225812    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:16:24.227114 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225814    2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:16:24.227114 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225818    2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:16:24.227114 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225821    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:16:24.227114 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225824    2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:16:24.227611 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225826    2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:16:24.227611 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225829    2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:16:24.227611 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225831    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:16:24.227611 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225834    2579 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:16:24.227611 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225837    2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:16:24.227611 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225840    2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:16:24.227611 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225843    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:16:24.227611 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225846    2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:16:24.227611 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225848    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:16:24.227611 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225851    2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:16:24.227611 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225855    2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:16:24.227611 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225858    2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:16:24.227611 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225861    2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:16:24.227611 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225864    2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:16:24.227611 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225867    2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:16:24.227611 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225870    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:16:24.227611 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225873    2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:16:24.227611 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225876    2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:16:24.227611 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225879    2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:16:24.228050 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225882    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:16:24.228050 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225885    2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:16:24.228050 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225889    2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:16:24.228050 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225891    2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:16:24.228050 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225894    2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:16:24.228050 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.225896    2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:16:24.228050 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226732    2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:16:24.228050 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226739    2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:16:24.228050 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226743    2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:16:24.228050 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226746    2579 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:16:24.228050 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226749    2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:16:24.228050 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226751    2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:16:24.228050 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226755    2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:16:24.228050 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226758    2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:16:24.228050 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226760    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:16:24.228050 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226763    2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:16:24.228050 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226767    2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:16:24.228050 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226769    2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:16:24.228050 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226772    2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:16:24.228050 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226776    2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:16:24.228536 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226779    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:16:24.228536 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226782    2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:16:24.228536 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226785    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:16:24.228536 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226787    2579 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:16:24.228536 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226790    2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:16:24.228536 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226793    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:16:24.228536 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226796    2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:16:24.228536 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226799    2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:16:24.228536 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226801    2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:16:24.228536 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226804    2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:16:24.228536 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226807    2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:16:24.228536 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226810    2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:16:24.228536 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226812    2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:16:24.228536 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226816    2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:16:24.228536 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226820    2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:16:24.228536 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226823    2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:16:24.228536 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226826    2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:16:24.228536 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226829    2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:16:24.228536 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226832    2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:16:24.228971 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226835    2579 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:16:24.228971 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226837    2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:16:24.228971 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226840    2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:16:24.228971 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226843    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:16:24.228971 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226846    2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:16:24.228971 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226849    2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:16:24.228971 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226852    2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:16:24.228971 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226855    2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:16:24.228971 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226858    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:16:24.228971 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226861    2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:16:24.228971 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226863    2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:16:24.228971 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226866    2579 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:16:24.228971 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226869    2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:16:24.228971 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226872    2579 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:16:24.228971 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226874    2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:16:24.228971 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226877    2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:16:24.228971 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226879    2579 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:16:24.228971 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226882    2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:16:24.228971 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226885    2579 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:16:24.228971 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226888    2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:16:24.229467 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226892    2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:16:24.229467 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226895    2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:16:24.229467 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226897    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:16:24.229467 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226900    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:16:24.229467 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226903    2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:16:24.229467 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226905    2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:16:24.229467 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226908    2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:16:24.229467 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226910    2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:16:24.229467 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226913    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:16:24.229467 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226916    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:16:24.229467 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226918    2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:16:24.229467 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226921    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:16:24.229467 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226924    2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:16:24.229467 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226926    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:16:24.229467 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226930    2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:16:24.229467 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226933    2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:16:24.229467 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226936    2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:16:24.229467 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226938    2579 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:16:24.229467 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226941    2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:16:24.229943 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226944    2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:16:24.229943 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226947    2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:16:24.229943 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226949    2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:16:24.229943 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226953    2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:16:24.229943 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226955    2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:16:24.229943 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226958    2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:16:24.229943 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226960    2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:16:24.229943 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226963    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:16:24.229943 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226966    2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:16:24.229943 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226968    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:16:24.229943 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226972    2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:16:24.229943 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226977    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:16:24.229943 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226980    2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:16:24.229943 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226983    2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:16:24.229943 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226986    2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:16:24.229943 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226989    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:16:24.229943 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226991    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:16:24.229943 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226994    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:16:24.229943 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.226997    2579 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:16:24.229943 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227000    2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:16:24.230442 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227002    2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:16:24.230442 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227005    2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:16:24.230442 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227007    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:16:24.230442 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227010    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:16:24.230442 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227013    2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:16:24.230442 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227016    2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:16:24.230442 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227018    2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:16:24.230442 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227021    2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:16:24.230442 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227024    2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:16:24.230442 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227027    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:16:24.230442 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227030    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:16:24.230442 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227033    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:16:24.230442 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227036    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:16:24.230442 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227039    2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:16:24.230442 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227566    2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:16:24.230442 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227572    2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:16:24.230442 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227575    2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:16:24.230442 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227578    2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:16:24.230868 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227580    2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:16:24.230868 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227583    2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:16:24.230868 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227586    2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:16:24.230868 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227588    2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:16:24.230868 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227591    2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:16:24.230868 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227593    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:16:24.230868 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227596    2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:16:24.230868 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227599    2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:16:24.230868 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227601    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:16:24.230868 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227604    2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:16:24.230868 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227607    2579 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:16:24.230868 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227610    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:16:24.230868 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227612    2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:16:24.230868 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227615    2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:16:24.230868 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227618    2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:16:24.230868 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227620    2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:16:24.230868 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227623    2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:16:24.230868 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227626    2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:16:24.230868 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227628    2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:16:24.230868 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227634    2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:16:24.231397 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227636    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:16:24.231397 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227639    2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:16:24.231397 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227642    2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:16:24.231397 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227645    2579 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:16:24.231397 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227648    2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:16:24.231397 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227650    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:16:24.231397 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227653    2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:16:24.231397 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227656    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:16:24.231397 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227659    2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:16:24.231397 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227661    2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:16:24.231397 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227664    2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:16:24.231397 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227666    2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:16:24.231397 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227669    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:16:24.231397 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227671    2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:16:24.231397 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227674    2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:16:24.231397 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227676    2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:16:24.231397 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227679    2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:16:24.231397 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227681    2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:16:24.231397 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227684    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:16:24.231836 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227686    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:16:24.231836 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227689    2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:16:24.231836 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227691    2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:16:24.231836 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227694    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:16:24.231836 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227696    2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:16:24.231836 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227700    2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:16:24.231836 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227704    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:16:24.231836 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227708    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:16:24.231836 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227711    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:16:24.231836 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227713    2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:16:24.231836 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227716    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:16:24.231836 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227718    2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:16:24.231836 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227722    2579 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:16:24.231836 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227725    2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:16:24.231836 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227728    2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:16:24.231836 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227731    2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:16:24.231836 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227733    2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:16:24.231836 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227736    2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:16:24.231836 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227738    2579 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:16:24.232287 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227741    2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:16:24.232287 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227743    2579 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:16:24.232287 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227746    2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:16:24.232287 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227749    2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:16:24.232287 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227751    2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:16:24.232287 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227754    2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:16:24.232287 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227756    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:16:24.232287 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227759    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:16:24.232287 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227762    2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:16:24.232287 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227764    2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:16:24.232287 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227767    2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:16:24.232287 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227769    2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:16:24.232287 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227772    2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:16:24.232287 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227774    2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:16:24.232287 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227777    2579 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:16:24.232287 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227779    2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:16:24.232287 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227781    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:16:24.232287 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227784    2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:16:24.232287 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227786    2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:16:24.232287 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227789    2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:16:24.232755 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227792    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:16:24.232755 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227794    2579 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:16:24.232755 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227797    2579 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:16:24.232755 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227799    2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:16:24.232755 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227802    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:16:24.232755 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227806    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:16:24.232755 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227809    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:16:24.232755 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227813    2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:16:24.232755 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227816    2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:16:24.232755 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227819    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:16:24.232755 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227821    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:16:24.232755 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227824    2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:16:24.232755 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227826    2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:16:24.232755 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227829    2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:16:24.232755 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227831    2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:16:24.232755 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227834    2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:16:24.232755 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227837    2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:16:24.232755 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227839    2579 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:16:24.232755 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227841    2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:16:24.232755 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227844    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:16:24.233243 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227847    2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:16:24.233243 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227849    2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:16:24.233243 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227853    2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:16:24.233243 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.227858    2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:16:24.233243 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.228957    2579 flags.go:64] FLAG: --address="0.0.0.0"
Nov 04 03:16:24.233243 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.228968    2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 04 03:16:24.233243 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.228976    2579 flags.go:64] FLAG: --anonymous-auth="true"
Nov 04 03:16:24.233243 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.228981    2579 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 04 03:16:24.233243 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.228986    2579 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 04 03:16:24.233243 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.228989    2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 04 03:16:24.233243 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.228994    2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 04 03:16:24.233243 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.228998    2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 04 03:16:24.233243 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229002    2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 04 03:16:24.233243 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229005    2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 04 03:16:24.233243 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229008    2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 04 03:16:24.233243 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229012    2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 04 03:16:24.233243 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229015    2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 04 03:16:24.233243 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229019    2579 flags.go:64] FLAG: --cgroup-root=""
Nov 04 03:16:24.233243 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229022    2579 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 04 03:16:24.233243 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229025    2579 flags.go:64] FLAG: --client-ca-file=""
Nov 04 03:16:24.233243 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229028    2579 flags.go:64] FLAG: --cloud-provider="external"
Nov 04 03:16:24.233243 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229032    2579 flags.go:64] FLAG: --cluster-dns="[]"
Nov 04 03:16:24.233243 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229036    2579 flags.go:64] FLAG: --cluster-domain=""
Nov 04 03:16:24.233790 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229038    2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 04 03:16:24.233790 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229041    2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 04 03:16:24.233790 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229045    2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 04 03:16:24.233790 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229048    2579 flags.go:64] FLAG: --container-log-max-files="5"
Nov 04 03:16:24.233790 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229053    2579 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 04 03:16:24.233790 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229056    2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 04 03:16:24.233790 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229059    2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 04 03:16:24.233790 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229074    2579 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 04 03:16:24.233790 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229078    2579 flags.go:64] FLAG: --contention-profiling="false"
Nov 04 03:16:24.233790 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229081    2579 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 04 03:16:24.233790 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229084    2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 04 03:16:24.233790 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229087    2579 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 04 03:16:24.233790 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229090    2579 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 04 03:16:24.233790 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229094    2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 04 03:16:24.233790 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229097    2579 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 04 03:16:24.233790 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229100    2579 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 04 03:16:24.233790 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229103    2579 flags.go:64] FLAG: --enable-load-reader="false"
Nov 04 03:16:24.233790 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229106    2579 flags.go:64] FLAG: --enable-server="true"
Nov 04 03:16:24.233790 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229109    2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 04 03:16:24.233790 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229114    2579 flags.go:64] FLAG: --event-burst="100"
Nov 04 03:16:24.233790 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229117    2579 flags.go:64] FLAG: --event-qps="50"
Nov 04 03:16:24.233790 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229121    2579 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 04 03:16:24.233790 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229123    2579 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 04 03:16:24.233790 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229126    2579 flags.go:64] FLAG: --eviction-hard=""
Nov 04 03:16:24.234422 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229131    2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 04 03:16:24.234422 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229134    2579 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 04 03:16:24.234422 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229137    2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 04 03:16:24.234422 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229140    2579 flags.go:64] FLAG: --eviction-soft=""
Nov 04 03:16:24.234422 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229144    2579 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 04 03:16:24.234422 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229147    2579 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 04 03:16:24.234422 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229150    2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 04 03:16:24.234422 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229153    2579 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 04 03:16:24.234422 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229156    2579 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 04 03:16:24.234422 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229159    2579 flags.go:64] FLAG: --fail-swap-on="true"
Nov 04 03:16:24.234422 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229162    2579 flags.go:64] FLAG: --feature-gates=""
Nov 04 03:16:24.234422 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229166    2579 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 04 03:16:24.234422 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229185    2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 04 03:16:24.234422 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229190    2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 04 03:16:24.234422 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229195    2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 04 03:16:24.234422 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229198    2579 flags.go:64] FLAG: --healthz-port="10248"
Nov 04 03:16:24.234422 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229201    2579 flags.go:64] FLAG: --help="false"
Nov 04 03:16:24.234422 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229204    2579 flags.go:64] FLAG: --hostname-override="ip-10-0-128-50.us-west-2.compute.internal"
Nov 04 03:16:24.234422 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229207    2579 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 04 03:16:24.234422 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229210    2579 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 04 03:16:24.234422 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229214    2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 04 03:16:24.234422 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229217    2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 04 03:16:24.234422 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229220    2579 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 04 03:16:24.234955 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229223    2579 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 04 03:16:24.234955 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229226    2579 flags.go:64] FLAG: --image-service-endpoint=""
Nov 04 03:16:24.234955 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229229    2579 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 04 03:16:24.234955 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229232    2579 flags.go:64] FLAG: --kube-api-burst="100"
Nov 04 03:16:24.234955 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229235    2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 04 03:16:24.234955 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229238    2579 flags.go:64] FLAG: --kube-api-qps="50"
Nov 04 03:16:24.234955 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229241    2579 flags.go:64] FLAG: --kube-reserved=""
Nov 04 03:16:24.234955 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229244    2579 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 04 03:16:24.234955 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229246    2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 04 03:16:24.234955 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229250    2579 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 04 03:16:24.234955 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229253    2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 04 03:16:24.234955 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229256    2579 flags.go:64] FLAG: --lock-file=""
Nov 04 03:16:24.234955 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229259    2579 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 04 03:16:24.234955 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229262    2579 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 04 03:16:24.234955 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229266    2579 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 04 03:16:24.234955 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229272    2579 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 04 03:16:24.234955 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229275    2579 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 04 03:16:24.234955 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229278    2579 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 04 03:16:24.234955 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229281    2579 flags.go:64] FLAG: --logging-format="text"
Nov 04 03:16:24.234955 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229284    2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 04 03:16:24.234955 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229288    2579 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 04 03:16:24.234955 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229291    2579 flags.go:64] FLAG: --manifest-url=""
Nov 04 03:16:24.234955 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229294    2579 flags.go:64] FLAG: --manifest-url-header=""
Nov 04 03:16:24.234955 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229298    2579 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 04 03:16:24.234955 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229302    2579 flags.go:64] FLAG: --max-open-files="1000000"
Nov 04 03:16:24.235594 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229306    2579 flags.go:64] FLAG: --max-pods="110"
Nov 04 03:16:24.235594 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229309    2579 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 04 03:16:24.235594 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229311    2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 04 03:16:24.235594 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229315    2579 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 04 03:16:24.235594 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229318    2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 04 03:16:24.235594 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229321    2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 04 03:16:24.235594 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229324    2579 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 04 03:16:24.235594 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229327    2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 04 03:16:24.235594 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229334    2579 flags.go:64] FLAG: --node-status-max-images="50"
Nov 04 03:16:24.235594 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229337    2579 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 04 03:16:24.235594 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229340    2579 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 04 03:16:24.235594 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229343    2579 flags.go:64] FLAG: --pod-cidr=""
Nov 04 03:16:24.235594 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229346    2579 flags.go:64] FLAG: --pod-manifest-path=""
Nov 04 03:16:24.235594 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229349    2579 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 04 03:16:24.235594 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229352    2579 flags.go:64] FLAG: --pods-per-core="0"
Nov 04 03:16:24.235594 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229355    2579 flags.go:64] FLAG: --port="10250"
Nov 04 03:16:24.235594 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229358    2579 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 04 03:16:24.235594 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229361    2579 flags.go:64] FLAG: --provider-id="aws:///us-west-2a/i-0bab39cfb9c95f313"
Nov 04 03:16:24.235594 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229365    2579 flags.go:64] FLAG: --qos-reserved=""
Nov 04 03:16:24.235594 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229368    2579 flags.go:64] FLAG: --read-only-port="10255"
Nov 04 03:16:24.235594 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229371    2579 flags.go:64] FLAG: --register-node="true"
Nov 04 03:16:24.235594 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229374    2579 flags.go:64] FLAG: --register-with-taints=""
Nov 04 03:16:24.235594 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229377    2579 flags.go:64] FLAG: --registry-burst="10"
Nov 04 03:16:24.235594 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229380    2579 flags.go:64] FLAG: --registry-qps="5"
Nov 04 03:16:24.235594 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229383    2579 flags.go:64] FLAG: --reserved-cpus=""
Nov 04 03:16:24.235594 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229386    2579 flags.go:64] FLAG: --reserved-memory=""
Nov 04 03:16:24.236406 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229390    2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 04 03:16:24.236406 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229393    2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 04 03:16:24.236406 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229396    2579 flags.go:64] FLAG: --rotate-certificates="false"
Nov 04 03:16:24.236406 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229399    2579 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 04 03:16:24.236406 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229402    2579 flags.go:64] FLAG: --runonce="false"
Nov 04 03:16:24.236406 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229405    2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 04 03:16:24.236406 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229409    2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 04 03:16:24.236406 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229412    2579 flags.go:64] FLAG: --seccomp-default="false"
Nov 04 03:16:24.236406 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229415    2579 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 04 03:16:24.236406 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229427    2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 04 03:16:24.236406 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229431    2579 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 04 03:16:24.236406 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229434    2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 04 03:16:24.236406 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229438    2579 flags.go:64] FLAG: --storage-driver-password="root"
Nov 04 03:16:24.236406 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229441    2579 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 04 03:16:24.236406 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229443    2579 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 04 03:16:24.236406 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229446    2579 flags.go:64] FLAG: --storage-driver-user="root"
Nov 04 03:16:24.236406 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229449    2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 04 03:16:24.236406 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229452    2579 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 04 03:16:24.236406 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229455    2579 flags.go:64] FLAG: --system-cgroups=""
Nov 04 03:16:24.236406 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229458    2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 04 03:16:24.236406 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229463    2579 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 04 03:16:24.236406 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229466    2579 flags.go:64] FLAG: --tls-cert-file=""
Nov 04 03:16:24.236406 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229469    2579 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 04 03:16:24.236406 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229473    2579 flags.go:64] FLAG: --tls-min-version=""
Nov 04 03:16:24.236406 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229478    2579 flags.go:64] FLAG: --tls-private-key-file=""
Nov 04 03:16:24.237326 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229481    2579 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 04 03:16:24.237326 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229484    2579 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 04 03:16:24.237326 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229487    2579 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 04 03:16:24.237326 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229490    2579 flags.go:64] FLAG: --v="2"
Nov 04 03:16:24.237326 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229495    2579 flags.go:64] FLAG: --version="false"
Nov 04 03:16:24.237326 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229499    2579 flags.go:64] FLAG: --vmodule=""
Nov 04 03:16:24.237326 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229503    2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 04 03:16:24.237326 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229507    2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 04 03:16:24.237326 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229666    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:16:24.237326 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229670    2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:16:24.237326 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229674    2579 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:16:24.237326 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229677    2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:16:24.237326 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229680    2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:16:24.237326 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229683    2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:16:24.237326 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229686    2579 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:16:24.237326 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229689    2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:16:24.237326 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229694    2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:16:24.237326 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229696    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:16:24.237326 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229699    2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:16:24.237326 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229702    2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:16:24.237326 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229704    2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:16:24.237844 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229707    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:16:24.237844 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229709    2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:16:24.237844 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229712    2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:16:24.237844 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229715    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:16:24.237844 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229717    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:16:24.237844 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229720    2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:16:24.237844 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229722    2579 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:16:24.237844 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229726    2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:16:24.237844 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229730    2579 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:16:24.237844 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229733    2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:16:24.237844 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229736    2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:16:24.237844 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229739    2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:16:24.237844 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229742    2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:16:24.237844 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229744    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:16:24.237844 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229747    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:16:24.237844 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229749    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:16:24.237844 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229752    2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:16:24.237844 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229755    2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:16:24.237844 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229757    2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:16:24.238319 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229760    2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:16:24.238319 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229762    2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:16:24.238319 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229764    2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:16:24.238319 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229767    2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:16:24.238319 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229770    2579 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:16:24.238319 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229772    2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:16:24.238319 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229775    2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:16:24.238319 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229779    2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:16:24.238319 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229784    2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:16:24.238319 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229787    2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:16:24.238319 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229789    2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:16:24.238319 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229792    2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:16:24.238319 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229794    2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:16:24.238319 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229797    2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:16:24.238319 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229799    2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:16:24.238319 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229802    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:16:24.238319 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229804    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:16:24.238319 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229807    2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:16:24.238319 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229810    2579 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:16:24.238319 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229812    2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:16:24.238790 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229815    2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:16:24.238790 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229817    2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:16:24.238790 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229820    2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:16:24.238790 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229824    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:16:24.238790 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229827    2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:16:24.238790 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229829    2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:16:24.238790 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229832    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:16:24.238790 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229835    2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:16:24.238790 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229837    2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:16:24.238790 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229840    2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:16:24.238790 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229843    2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:16:24.238790 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229845    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:16:24.238790 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229848    2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:16:24.238790 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229851    2579 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:16:24.238790 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229854    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:16:24.238790 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229856    2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:16:24.238790 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229858    2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:16:24.238790 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229861    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:16:24.238790 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229864    2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:16:24.239367 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229866    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:16:24.239367 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229871    2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:16:24.239367 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229874    2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:16:24.239367 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229876    2579 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:16:24.239367 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229879    2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:16:24.239367 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229881    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:16:24.239367 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229884    2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:16:24.239367 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229886    2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:16:24.239367 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229889    2579 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:16:24.239367 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229891    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:16:24.239367 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229894    2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:16:24.239367 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229897    2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:16:24.239367 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229899    2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:16:24.239367 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229902    2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:16:24.239367 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229904    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:16:24.239367 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229907    2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:16:24.239367 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229911    2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:16:24.239367 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229913    2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:16:24.239367 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229916    2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:16:24.239820 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229918    2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:16:24.239820 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229921    2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:16:24.239820 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229924    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:16:24.239820 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229926    2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:16:24.239820 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229929    2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:16:24.239820 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229932    2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:16:24.239820 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229934    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:16:24.239820 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229937    2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:16:24.239820 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229939    2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:16:24.239820 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229942    2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:16:24.239820 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229944    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:16:24.239820 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229948    2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:16:24.239820 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229950    2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:16:24.239820 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229953    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:16:24.239820 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229957    2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:16:24.239820 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.229960    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:16:24.240207 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.229986    2579 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 03:16:24.240207 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230023    2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:16:24.240207 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230027    2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:16:24.240207 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230031    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:16:24.240207 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230034    2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:16:24.240207 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230036    2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:16:24.240207 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230039    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:16:24.240207 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230043    2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:16:24.240207 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230045    2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:16:24.240207 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230048    2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:16:24.240207 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230051    2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:16:24.240207 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230054    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:16:24.240207 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230057    2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:16:24.240207 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230060    2579 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:16:24.240207 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230075    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:16:24.240207 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230078    2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:16:24.240604 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230081    2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:16:24.240604 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230084    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:16:24.240604 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230087    2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:16:24.240604 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230089    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:16:24.240604 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230092    2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:16:24.240604 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230095    2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:16:24.240604 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230097    2579 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:16:24.240604 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230100    2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:16:24.240604 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230103    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:16:24.240604 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230105    2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:16:24.240604 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230108    2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:16:24.240604 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230111    2579 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:16:24.240604 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230113    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:16:24.240604 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230116    2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:16:24.240604 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230120    2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:16:24.240604 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230123    2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:16:24.240604 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230126    2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:16:24.240604 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230129    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:16:24.240604 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230131    2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:16:24.241043 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230134    2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:16:24.241043 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230136    2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:16:24.241043 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230139    2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:16:24.241043 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230141    2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:16:24.241043 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230144    2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:16:24.241043 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230146    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:16:24.241043 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230149    2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:16:24.241043 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230152    2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:16:24.241043 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230154    2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:16:24.241043 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230157    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:16:24.241043 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230159    2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:16:24.241043 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230163    2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:16:24.241043 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230166    2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:16:24.241043 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230168    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:16:24.241043 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230171    2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:16:24.241043 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230174    2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:16:24.241043 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230176    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:16:24.241043 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230179    2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:16:24.241043 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230181    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:16:24.241524 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230185    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:16:24.241524 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230187    2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:16:24.241524 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230190    2579 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:16:24.241524 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230193    2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:16:24.241524 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230195    2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:16:24.241524 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230198    2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:16:24.241524 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230201    2579 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:16:24.241524 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230203    2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:16:24.241524 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230207    2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:16:24.241524 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230210    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:16:24.241524 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230213    2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:16:24.241524 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230216    2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:16:24.241524 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230218    2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:16:24.241524 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230221    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:16:24.241524 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230223    2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:16:24.241524 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230226    2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:16:24.241524 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230228    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:16:24.241524 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230231    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:16:24.241524 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230233    2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:16:24.241963 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230236    2579 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:16:24.241963 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230240    2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:16:24.241963 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230244    2579 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:16:24.241963 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230246    2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:16:24.241963 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230249    2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:16:24.241963 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230253    2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:16:24.241963 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230255    2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:16:24.241963 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230258    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:16:24.241963 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230261    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:16:24.241963 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230263    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:16:24.241963 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230266    2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:16:24.241963 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230268    2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:16:24.241963 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230271    2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:16:24.241963 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230274    2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:16:24.241963 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230276    2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:16:24.241963 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230279    2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:16:24.241963 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230281    2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:16:24.241963 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230284    2579 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:16:24.241963 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230286    2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:16:24.241963 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230289    2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:16:24.242500 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230293    2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:16:24.242500 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230297    2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:16:24.242500 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230300    2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:16:24.242500 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230303    2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:16:24.242500 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230306    2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:16:24.242500 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230309    2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:16:24.242500 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230312    2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:16:24.242500 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230315    2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:16:24.242500 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230317    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:16:24.242500 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230320    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:16:24.242500 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230322    2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:16:24.242500 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230326    2579 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:16:24.242500 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230328    2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:16:24.242500 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.230331    2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:16:24.242500 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.238467    2579 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 04 03:16:24.242500 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.238484    2579 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 04 03:16:24.242500 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.238507    2579 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 04 03:16:24.242500 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.238513    2579 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 04 03:16:24.242500 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238573    2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:16:24.242500 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238577    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:16:24.242984 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238580    2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:16:24.242984 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238583    2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:16:24.242984 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238587    2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:16:24.242984 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238590    2579 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:16:24.242984 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238592    2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:16:24.242984 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238595    2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:16:24.242984 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238598    2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:16:24.242984 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238600    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:16:24.242984 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238603    2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:16:24.242984 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238606    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:16:24.242984 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238608    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:16:24.242984 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238611    2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:16:24.242984 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238613    2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:16:24.242984 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238616    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:16:24.242984 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238619    2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:16:24.242984 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238622    2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:16:24.242984 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238624    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:16:24.242984 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238627    2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:16:24.242984 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238630    2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:16:24.243488 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238632    2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:16:24.243488 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238636    2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:16:24.243488 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238638    2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:16:24.243488 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238641    2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:16:24.243488 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238644    2579 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:16:24.243488 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238647    2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:16:24.243488 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238649    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:16:24.243488 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238652    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:16:24.243488 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238655    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:16:24.243488 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238657    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:16:24.243488 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238660    2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:16:24.243488 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238662    2579 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:16:24.243488 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238665    2579 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:16:24.243488 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238668    2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:16:24.243488 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238670    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:16:24.243488 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238673    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:16:24.243488 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238675    2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:16:24.243488 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238677    2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:16:24.243488 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238680    2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:16:24.244049 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238682    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:16:24.244049 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238685    2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:16:24.244049 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238688    2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:16:24.244049 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238690    2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:16:24.244049 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238692    2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:16:24.244049 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238695    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:16:24.244049 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238697    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:16:24.244049 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238700    2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:16:24.244049 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238703    2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:16:24.244049 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238705    2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:16:24.244049 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238708    2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:16:24.244049 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238711    2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:16:24.244049 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238714    2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:16:24.244049 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238717    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:16:24.244049 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238719    2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:16:24.244049 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238722    2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:16:24.244049 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238725    2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:16:24.244049 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238727    2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:16:24.244049 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238731    2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:16:24.244519 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238735    2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:16:24.244519 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238738    2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:16:24.244519 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238741    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:16:24.244519 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238745    2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:16:24.244519 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238750    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:16:24.244519 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238753    2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:16:24.244519 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238756    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:16:24.244519 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238758    2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:16:24.244519 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238761    2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:16:24.244519 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238764    2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:16:24.244519 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238766    2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:16:24.244519 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238769    2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:16:24.244519 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238771    2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:16:24.244519 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238774    2579 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:16:24.244519 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238777    2579 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:16:24.244519 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238779    2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:16:24.244519 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238782    2579 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:16:24.244519 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238784    2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:16:24.244519 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238787    2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:16:24.244519 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238789    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:16:24.245054 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238793    2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:16:24.245054 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238796    2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:16:24.245054 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238799    2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:16:24.245054 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238802    2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:16:24.245054 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238805    2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:16:24.245054 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238807    2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:16:24.245054 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238810    2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:16:24.245054 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238812    2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:16:24.245054 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238815    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:16:24.245054 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238818    2579 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:16:24.245054 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238820    2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:16:24.245054 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238823    2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:16:24.245054 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238825    2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:16:24.245054 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238828    2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:16:24.245054 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238830    2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:16:24.245054 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238833    2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:16:24.245054 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238836    2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:16:24.245054 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238838    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:16:24.245054 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238841    2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:16:24.245054 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238843    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:16:24.245623 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238846    2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:16:24.245623 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238848    2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:16:24.245623 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238851    2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:16:24.245623 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238854    2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:16:24.245623 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238856    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:16:24.245623 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238859    2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:16:24.245623 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.238861    2579 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:16:24.245623 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.238888    2579 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 03:16:24.245623 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239027    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:16:24.245623 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239033    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:16:24.245623 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239036    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:16:24.245623 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239039    2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:16:24.245623 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239042    2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:16:24.245623 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239046    2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:16:24.245623 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239048    2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:16:24.245985 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239051    2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:16:24.245985 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239054    2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:16:24.245985 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239056    2579 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:16:24.245985 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239059    2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:16:24.245985 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239077    2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:16:24.245985 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239082    2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:16:24.245985 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239085    2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:16:24.245985 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239087    2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:16:24.245985 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239090    2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:16:24.245985 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239093    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:16:24.245985 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239095    2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:16:24.245985 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239098    2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:16:24.245985 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239101    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:16:24.245985 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239103    2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:16:24.245985 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239105    2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:16:24.245985 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239108    2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:16:24.245985 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239111    2579 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:16:24.245985 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239115    2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:16:24.245985 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239118    2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:16:24.246625 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239121    2579 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:16:24.246625 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239126    2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:16:24.246625 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239129    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:16:24.246625 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239132    2579 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:16:24.246625 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239135    2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:16:24.246625 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239138    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:16:24.246625 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239140    2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:16:24.246625 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239143    2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:16:24.246625 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239146    2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:16:24.246625 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239148    2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:16:24.246625 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239151    2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:16:24.246625 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239153    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:16:24.246625 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239157    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:16:24.246625 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239159    2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:16:24.246625 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239162    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:16:24.246625 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239166    2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:16:24.246625 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239169    2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:16:24.246625 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239172    2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:16:24.246625 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239175    2579 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:16:24.247414 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239178    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:16:24.247414 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239180    2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:16:24.247414 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239183    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:16:24.247414 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239185    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:16:24.247414 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239188    2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:16:24.247414 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239191    2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:16:24.247414 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239193    2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:16:24.247414 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239196    2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:16:24.247414 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239199    2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:16:24.247414 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239201    2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:16:24.247414 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239204    2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:16:24.247414 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239206    2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:16:24.247414 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239209    2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:16:24.247414 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239212    2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:16:24.247414 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239215    2579 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:16:24.247414 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239217    2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:16:24.247414 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239220    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:16:24.247414 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239223    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:16:24.247414 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239225    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:16:24.247872 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239228    2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:16:24.247872 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239231    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:16:24.247872 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239233    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:16:24.247872 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239236    2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:16:24.247872 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239238    2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:16:24.247872 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239241    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:16:24.247872 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239244    2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:16:24.247872 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239247    2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:16:24.247872 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239250    2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:16:24.247872 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239255    2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:16:24.247872 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239258    2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:16:24.247872 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239260    2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:16:24.247872 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239263    2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:16:24.247872 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239266    2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:16:24.247872 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239268    2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:16:24.247872 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239271    2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:16:24.247872 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239273    2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:16:24.247872 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239276    2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:16:24.247872 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239279    2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:16:24.247872 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239281    2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:16:24.248439 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239284    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:16:24.248439 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239287    2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:16:24.248439 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239289    2579 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:16:24.248439 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239291    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:16:24.248439 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239294    2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:16:24.248439 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239297    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:16:24.248439 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239299    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:16:24.248439 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239302    2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:16:24.248439 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239304    2579 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:16:24.248439 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239307    2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:16:24.248439 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239309    2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:16:24.248439 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239312    2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:16:24.248439 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239314    2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:16:24.248439 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239317    2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:16:24.248439 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239319    2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:16:24.248439 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239322    2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:16:24.248439 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239324    2579 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:16:24.248439 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239327    2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:16:24.248439 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239329    2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:16:24.248439 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239332    2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:16:24.248899 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239335    2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:16:24.248899 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239343    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:16:24.248899 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.239366    2579 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 03:16:24.248899 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239398    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:16:24.248899 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239403    2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:16:24.248899 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239406    2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:16:24.248899 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239409    2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:16:24.248899 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239412    2579 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:16:24.248899 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239415    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:16:24.248899 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239418    2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:16:24.248899 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239420    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:16:24.248899 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239423    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:16:24.248899 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239426    2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:16:24.248899 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239428    2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:16:24.248899 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239431    2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:16:24.248899 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239433    2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:16:24.249501 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239436    2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:16:24.249501 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239439    2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:16:24.249501 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239441    2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:16:24.249501 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239444    2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:16:24.249501 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239446    2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:16:24.249501 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239449    2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:16:24.249501 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239451    2579 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:16:24.249501 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239454    2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:16:24.249501 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239457    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:16:24.249501 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239459    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:16:24.249501 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239462    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:16:24.249501 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239464    2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:16:24.249501 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239467    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:16:24.249501 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239470    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:16:24.249501 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239472    2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:16:24.249501 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239475    2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:16:24.249501 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239478    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:16:24.249501 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239482    2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:16:24.249501 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239486    2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:16:24.249501 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239489    2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:16:24.250154 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239491    2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:16:24.250154 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239494    2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:16:24.250154 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239497    2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:16:24.250154 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239499    2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:16:24.250154 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239502    2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:16:24.250154 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239505    2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:16:24.250154 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239507    2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:16:24.250154 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239510    2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:16:24.250154 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239513    2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:16:24.250154 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239515    2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:16:24.250154 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239518    2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:16:24.250154 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239521    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:16:24.250154 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239524    2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:16:24.250154 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239527    2579 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:16:24.250154 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239529    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:16:24.250154 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239532    2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:16:24.250154 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239534    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:16:24.250154 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239537    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:16:24.250154 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239540    2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:16:24.250154 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239543    2579 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:16:24.250628 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239546    2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:16:24.250628 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239548    2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:16:24.250628 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239550    2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:16:24.250628 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239553    2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:16:24.250628 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239556    2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:16:24.250628 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239558    2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:16:24.250628 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239561    2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:16:24.250628 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239563    2579 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:16:24.250628 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239566    2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:16:24.250628 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239569    2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:16:24.250628 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239574    2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:16:24.250628 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239577    2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:16:24.250628 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239579    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:16:24.250628 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239582    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:16:24.250628 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239585    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:16:24.250628 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239587    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:16:24.250628 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239590    2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:16:24.250628 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239593    2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:16:24.250628 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239595    2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:16:24.250628 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239598    2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:16:24.251127 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239601    2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:16:24.251127 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239604    2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:16:24.251127 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239606    2579 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:16:24.251127 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239609    2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:16:24.251127 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239611    2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:16:24.251127 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239614    2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:16:24.251127 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239617    2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:16:24.251127 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239619    2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:16:24.251127 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239622    2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:16:24.251127 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239625    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:16:24.251127 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239627    2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:16:24.251127 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239630    2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:16:24.251127 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239632    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:16:24.251127 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239635    2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:16:24.251127 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239637    2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:16:24.251127 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239640    2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:16:24.251127 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239642    2579 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:16:24.251127 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239646    2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:16:24.251127 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239649    2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:16:24.251615 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239653    2579 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:16:24.251615 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239658    2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:16:24.251615 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239664    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:16:24.251615 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239668    2579 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:16:24.251615 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239672    2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:16:24.251615 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239675    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:16:24.251615 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239677    2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:16:24.251615 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239680    2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:16:24.251615 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239683    2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:16:24.251615 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239686    2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:16:24.251615 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239688    2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:16:24.251615 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239691    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:16:24.251615 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239693    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:16:24.251615 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:24.239696    2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:16:24.251615 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.239753    2579 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 04 03:16:24.251615 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.242031    2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Nov 04 03:16:24.251615 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.243491    2579 server.go:1014] "Starting client certificate rotation"
Nov 04 03:16:24.251615 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.243607    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 04 03:16:24.252173 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.243660    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 04 03:16:24.252173 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.245788    2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 03:16:24.252173 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.248985    2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 03:16:24.261781 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.261759    2579 log.go:25] "Validated CRI v1 runtime API"
Nov 04 03:16:24.266202 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.266188    2579 log.go:25] "Validated CRI v1 image API"
Nov 04 03:16:24.267033 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.267011    2579 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 04 03:16:24.269918 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.269896    2579 fs.go:135] Filesystem UUIDs: map[008627af-74db-4dc7-968a-2fa6fb3712dc:/dev/nvme0n1p4 1a774b2d-4b73-49a9-be43-014b6c77e293:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Nov 04 03:16:24.270000 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.269918    2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Nov 04 03:16:24.276073 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.275939    2579 manager.go:217] Machine: {Timestamp:2026-11-04 03:16:24.274370633 +0000 UTC m=+0.362678737 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099580 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ed8b0b8838639e66c5736906615ef SystemUUID:ec2ed8b0-b883-8639-e66c-5736906615ef BootID:1d9cfbae-33ed-4515-b116-6962f7c263ff Filesystems:[{Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:06:ff:c7:79:1b:c3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:06:ff:c7:79:1b:c3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ea:52:c8:85:ad:0f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 04 03:16:24.276906 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.276893    2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 04 03:16:24.277050 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.277034    2579 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 04 03:16:24.281801 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.281768    2579 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 04 03:16:24.281935 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.281801    2579 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-50.us-west-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 04 03:16:24.282621 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.282610    2579 topology_manager.go:143] "Creating topology manager with none policy"
Nov 04 03:16:24.282660 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.282623    2579 container_manager_linux.go:311] "Creating device plugin manager"
Nov 04 03:16:24.282660 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.282650    2579 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 03:16:24.282711 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.282663    2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 03:16:24.283617 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.283606    2579 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 04 03:16:24.287820 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.287808    2579 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 04 03:16:24.287934 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.287925    2579 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 04 03:16:24.288999 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.288988    2579 kubelet.go:493] "Attempting to sync node with API server"
Nov 04 03:16:24.289030 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.289004    2579 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 04 03:16:24.289030 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.289014    2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 04 03:16:24.289030 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.289023    2579 kubelet.go:405] "Adding apiserver pod source"
Nov 04 03:16:24.289138 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.289031    2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 04 03:16:24.289401 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.289387    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 03:16:24.291847 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.291832    2579 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 04 03:16:24.297705 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.297689    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 03:16:24.298550 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.298536    2579 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 04 03:16:24.298625 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.298576    2579 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 04 03:16:24.298771 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.298758    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 04 03:16:24.298828 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.298781    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 04 03:16:24.298828 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.298792    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 04 03:16:24.298828 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.298801    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 04 03:16:24.298828 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.298811    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 04 03:16:24.298828 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.298820    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 04 03:16:24.299478 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.299468    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 04 03:16:24.299526 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.299482    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 04 03:16:24.299526 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.299494    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 04 03:16:24.299526 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.299505    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 04 03:16:24.299526 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.299517    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 04 03:16:24.299769 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.299532    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 04 03:16:24.300148 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.300137    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 04 03:16:24.300205 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.300153    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 04 03:16:24.304030 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.304015    2579 server.go:1263] "Started kubelet"
Nov 04 03:16:24.304154 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.304097    2579 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 04 03:16:24.304200 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.304120    2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 04 03:16:24.304248 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.304216    2579 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 04 03:16:24.304829 ip-10-0-128-50 systemd[1]: Started Kubernetes Kubelet.
Nov 04 03:16:24.305853 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.305593    2579 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 04 03:16:24.308229 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.308216    2579 server.go:317] "Adding debug handlers to kubelet server"
Nov 04 03:16:24.310857 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:24.310824    2579 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Nov 04 03:16:24.312646 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.312625    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 04 03:16:24.313290 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.313273    2579 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 04 03:16:24.313442 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:24.313409    2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-128-50.us-west-2.compute.internal\" not found"
Nov 04 03:16:24.314109 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.314090    2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 04 03:16:24.314109 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.314090    2579 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 04 03:16:24.314245 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.314124    2579 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 04 03:16:24.314245 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.314176    2579 reconstruct.go:97] "Volume reconstruction finished"
Nov 04 03:16:24.314245 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.314183    2579 reconciler.go:29] "Reconciler: start to sync state"
Nov 04 03:16:24.314856 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.314841    2579 factory.go:55] Registering systemd factory
Nov 04 03:16:24.314856 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.314857    2579 factory.go:223] Registration of the systemd container factory successfully
Nov 04 03:16:24.315110 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.315094    2579 factory.go:153] Registering CRI-O factory
Nov 04 03:16:24.315175 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.315113    2579 factory.go:223] Registration of the crio container factory successfully
Nov 04 03:16:24.315239 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.315188    2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 04 03:16:24.315239 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.315205    2579 factory.go:103] Registering Raw factory
Nov 04 03:16:24.315239 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.315216    2579 manager.go:1199] Started watching for new ooms in manager
Nov 04 03:16:24.316275 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.316260    2579 manager.go:319] Starting recovery of all containers
Nov 04 03:16:24.328163 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.328143    2579 manager.go:324] Recovery completed
Nov 04 03:16:24.332608 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.332595    2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 03:16:24.334292 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.334275    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-50.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 03:16:24.334369 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.334303    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-50.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 03:16:24.334369 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.334313    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-50.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 03:16:24.334827 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.334815    2579 cpu_manager.go:226] "Starting" policy="none"
Nov 04 03:16:24.334870 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.334828    2579 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 04 03:16:24.334870 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.334817    2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 03:16:24.334870 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.334849    2579 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 04 03:16:24.336644 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.336625    2579 policy_none.go:50] "Start"
Nov 04 03:16:24.336711 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.336649    2579 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 04 03:16:24.336711 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.336665    2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 04 03:16:24.337570 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.337553    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-50.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 03:16:24.337618 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.337588    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-50.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 03:16:24.337618 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.337605    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-50.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 03:16:24.338095 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.338081    2579 policy_none.go:44] "Start"
Nov 04 03:16:24.377279 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.377251    2579 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 04 03:16:24.379940 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.379925    2579 manager.go:341] "Starting Device Plugin manager"
Nov 04 03:16:24.380585 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:24.380561    2579 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 04 03:16:24.380585 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.380581    2579 server.go:87] "Starting device plugin registration server"
Nov 04 03:16:24.380795 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.380782    2579 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 04 03:16:24.380867 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.380795    2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 04 03:16:24.380942 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.380927    2579 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 04 03:16:24.381017 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.381005    2579 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 04 03:16:24.381078 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.381017    2579 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 04 03:16:24.381506 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:24.381488    2579 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 04 03:16:24.381588 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:24.381535    2579 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-50.us-west-2.compute.internal\" not found"
Nov 04 03:16:24.454396 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.454369    2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-50.us-west-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Nov 04 03:16:24.455576 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:24.454372    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-50.us-west-2.compute.internal.18e4b933eef7cedc  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-50.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-50.us-west-2.compute.internal,},FirstTimestamp:2026-11-04 03:16:24.303988444 +0000 UTC m=+0.392296548,LastTimestamp:2026-11-04 03:16:24.303988444 +0000 UTC m=+0.392296548,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-50.us-west-2.compute.internal,}"
Nov 04 03:16:24.475287 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.475232    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 04 03:16:24.476449 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.476426    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 04 03:16:24.476547 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.476452    2579 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 04 03:16:24.476547 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.476474    2579 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 04 03:16:24.476547 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:24.476516    2579 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Nov 04 03:16:24.481000 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.480979    2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 03:16:24.481744 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.481728    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-50.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 03:16:24.481790 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.481762    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-50.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 03:16:24.481790 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.481778    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-50.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 03:16:24.481852 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.481810    2579 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-128-50.us-west-2.compute.internal"
Nov 04 03:16:24.504214 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:24.504188    2579 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-50.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Nov 04 03:16:24.557080 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.557045    2579 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 04 03:16:24.557157 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.557111    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-h7n6v"
Nov 04 03:16:24.577255 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.577236    2579 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.us-west-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-50.us-west-2.compute.internal"]
Nov 04 03:16:24.577330 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.577296    2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 03:16:24.578212 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.578198    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-50.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 03:16:24.578268 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.578226    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-50.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 03:16:24.578268 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.578236    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-50.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 03:16:24.579709 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.579697    2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 03:16:24.579837 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.579824    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.us-west-2.compute.internal"
Nov 04 03:16:24.579873 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.579858    2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 03:16:24.580488 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.580471    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-50.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 03:16:24.580578 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.580498    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-50.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 03:16:24.580578 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.580508    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-50.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 03:16:24.580578 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.580480    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-50.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 03:16:24.580678 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.580581    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-50.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 03:16:24.580678 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.580597    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-50.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 03:16:24.582140 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.582127    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-50.us-west-2.compute.internal"
Nov 04 03:16:24.582192 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.582150    2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 03:16:24.582932 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.582916    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-50.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 03:16:24.583007 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.582949    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-50.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 03:16:24.583007 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.582965    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-50.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 03:16:24.604038 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:24.604011    2579 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-50.us-west-2.compute.internal\" not found" node="ip-10-0-128-50.us-west-2.compute.internal"
Nov 04 03:16:24.607797 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:24.607779    2579 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-50.us-west-2.compute.internal\" not found" node="ip-10-0-128-50.us-west-2.compute.internal"
Nov 04 03:16:24.615820 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.615802    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2f2b595dbb5c94185c06c3dd3c0c8dce-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-50.us-west-2.compute.internal\" (UID: \"2f2b595dbb5c94185c06c3dd3c0c8dce\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.us-west-2.compute.internal"
Nov 04 03:16:24.615899 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.615832    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2f2b595dbb5c94185c06c3dd3c0c8dce-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-50.us-west-2.compute.internal\" (UID: \"2f2b595dbb5c94185c06c3dd3c0c8dce\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.us-west-2.compute.internal"
Nov 04 03:16:24.615899 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.615859    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/aeb0f7512125881a2a5e1813eb42a8f7-config\") pod \"kube-apiserver-proxy-ip-10-0-128-50.us-west-2.compute.internal\" (UID: \"aeb0f7512125881a2a5e1813eb42a8f7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-50.us-west-2.compute.internal"
Nov 04 03:16:24.626211 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.626192    2579 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 03:16:24.628204 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.628190    2579 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 03:16:24.716380 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.716347    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2f2b595dbb5c94185c06c3dd3c0c8dce-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-50.us-west-2.compute.internal\" (UID: \"2f2b595dbb5c94185c06c3dd3c0c8dce\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.us-west-2.compute.internal"
Nov 04 03:16:24.716380 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.716383    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/aeb0f7512125881a2a5e1813eb42a8f7-config\") pod \"kube-apiserver-proxy-ip-10-0-128-50.us-west-2.compute.internal\" (UID: \"aeb0f7512125881a2a5e1813eb42a8f7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-50.us-west-2.compute.internal"
Nov 04 03:16:24.716577 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.716407    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2f2b595dbb5c94185c06c3dd3c0c8dce-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-50.us-west-2.compute.internal\" (UID: \"2f2b595dbb5c94185c06c3dd3c0c8dce\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.us-west-2.compute.internal"
Nov 04 03:16:24.716577 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.716457    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2f2b595dbb5c94185c06c3dd3c0c8dce-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-50.us-west-2.compute.internal\" (UID: \"2f2b595dbb5c94185c06c3dd3c0c8dce\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.us-west-2.compute.internal"
Nov 04 03:16:24.716577 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.716475    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/aeb0f7512125881a2a5e1813eb42a8f7-config\") pod \"kube-apiserver-proxy-ip-10-0-128-50.us-west-2.compute.internal\" (UID: \"aeb0f7512125881a2a5e1813eb42a8f7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-50.us-west-2.compute.internal"
Nov 04 03:16:24.716577 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.716475    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2f2b595dbb5c94185c06c3dd3c0c8dce-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-50.us-west-2.compute.internal\" (UID: \"2f2b595dbb5c94185c06c3dd3c0c8dce\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.us-west-2.compute.internal"
Nov 04 03:16:24.730212 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.730137    2579 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 03:16:24.731475 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.731456    2579 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 03:16:24.732227 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.732202    2579 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-128-50.us-west-2.compute.internal"
Nov 04 03:16:24.732340 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:24.732227    2579 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-50.us-west-2.compute.internal\": node \"ip-10-0-128-50.us-west-2.compute.internal\" not found"
Nov 04 03:16:24.813920 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.813893    2579 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.us-west-2.compute.internal"
Nov 04 03:16:24.907835 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.907811    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.us-west-2.compute.internal"
Nov 04 03:16:24.912280 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:24.911972    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-50.us-west-2.compute.internal"
Nov 04 03:16:26.211417 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.211252    2579 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 03:16:26.211540 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.211518    2579 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-50.us-west-2.compute.internal"
Nov 04 03:16:26.271058 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.270988    2579 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 03:16:26.502010 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.501977    2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Nov 04 03:16:26.502519 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.502155    2579 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Nov 04 03:16:26.502519 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.502188    2579 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Nov 04 03:16:26.502519 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.502189    2579 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Nov 04 03:16:26.502519 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:26.502208    2579 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://acd60e97cb07342d6a02bc86b8f04d90-7f8ad830d0f07e24.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-128-50.us-west-2.compute.internal.18e4b933f0c670b6\": read tcp 10.0.128.50:57238->13.184.36.23:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-128-50.us-west-2.compute.internal.18e4b933f0c670b6  default   4850 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-50.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-50.us-west-2.compute.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-50.us-west-2.compute.internal,},FirstTimestamp:2026-11-04 03:16:24 +0000 UTC,LastTimestamp:2026-11-04 03:16:24.481770361 +0000 UTC m=+0.570078472,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-50.us-west-2.compute.internal,}"
Nov 04 03:16:26.549093 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.549025    2579 apiserver.go:52] "Watching apiserver"
Nov 04 03:16:26.572913 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.572886    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Nov 04 03:16:26.793497 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.793466    2579 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 04 03:16:26.793794 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.793775    2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-n4b5s","openshift-network-operator/iptables-alerter-sdhv4","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.us-west-2.compute.internal","openshift-multus/multus-9xgrv","openshift-network-diagnostics/network-check-target-hjssc","kube-system/global-pull-secret-syncer-tkkhk","kube-system/konnectivity-agent-m2vkr","kube-system/kube-apiserver-proxy-ip-10-0-128-50.us-west-2.compute.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8","openshift-dns/node-resolver-6pwhj","openshift-multus/multus-additional-cni-plugins-67bjk","openshift-multus/network-metrics-daemon-lngrr","openshift-ovn-kubernetes/ovnkube-node-vmw5z","openshift-cluster-node-tuning-operator/tuned-qtt77"]
Nov 04 03:16:26.795591 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.795573    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6pwhj"
Nov 04 03:16:26.796792 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.796769    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-m2vkr"
Nov 04 03:16:26.798167 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.798151    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lngrr"
Nov 04 03:16:26.798235 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:26.798217    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lngrr" podUID="fc3d3c0e-2877-4264-93a7-be3756d2dc5c"
Nov 04 03:16:26.799462 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.799418    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-n4b5s"
Nov 04 03:16:26.800646 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.800632    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.801808 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.801790    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkkhk"
Nov 04 03:16:26.801898 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:26.801850    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkkhk" podUID="d10e9268-df07-428b-ae25-528d9fe075d2"
Nov 04 03:16:26.803179 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.803166    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8"
Nov 04 03:16:26.804604 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.804591    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hjssc"
Nov 04 03:16:26.804657 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:26.804635    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hjssc" podUID="04b955ef-528b-4573-b18f-0957658f79e4"
Nov 04 03:16:26.805877 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.805857    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-sdhv4"
Nov 04 03:16:26.807335 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.807318    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:26.809500 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.809135    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-67bjk"
Nov 04 03:16:26.811482 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.811457    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:16:26.820092 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.820057    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-03 03:11:25 +0000 UTC" deadline="2028-04-20 00:40:07.203143619 +0000 UTC"
Nov 04 03:16:26.820092 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.820090    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12789h23m40.383055923s"
Nov 04 03:16:26.844809 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.844789    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 04 03:16:26.844891 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.844853    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 04 03:16:26.844953 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.844941    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-h66cx\""
Nov 04 03:16:26.845813 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.845796    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 04 03:16:26.845873 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.845822    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 04 03:16:26.845873 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.845869    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-vpndz\""
Nov 04 03:16:26.849067 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.849049    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 04 03:16:26.849154 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.849067    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 04 03:16:26.849218 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.849165    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 04 03:16:26.849218 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.849165    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-z2nst\""
Nov 04 03:16:26.849301 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.849240    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 04 03:16:26.849533 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.849517    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 04 03:16:26.849859 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.849845    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 04 03:16:26.849939 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.849926    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 04 03:16:26.849973 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.849947    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-56wj9\""
Nov 04 03:16:26.850260 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.850250    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 04 03:16:26.853047 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.853032    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 04 03:16:26.874678 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.874659    2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 04 03:16:26.880095 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.880078    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5cdl\" (UniqueName: \"kubernetes.io/projected/d604b334-07f4-40f2-bfa9-4b26c301c84c-kube-api-access-n5cdl\") pod \"aws-ebs-csi-driver-node-jvtr8\" (UID: \"d604b334-07f4-40f2-bfa9-4b26c301c84c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8"
Nov 04 03:16:26.880163 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.880102    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc3d3c0e-2877-4264-93a7-be3756d2dc5c-metrics-certs\") pod \"network-metrics-daemon-lngrr\" (UID: \"fc3d3c0e-2877-4264-93a7-be3756d2dc5c\") " pod="openshift-multus/network-metrics-daemon-lngrr"
Nov 04 03:16:26.880163 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.880120    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7713890e-763d-4a51-bffb-6cf8dd12fd18-host\") pod \"node-ca-n4b5s\" (UID: \"7713890e-763d-4a51-bffb-6cf8dd12fd18\") " pod="openshift-image-registry/node-ca-n4b5s"
Nov 04 03:16:26.880163 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.880143    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7713890e-763d-4a51-bffb-6cf8dd12fd18-serviceca\") pod \"node-ca-n4b5s\" (UID: \"7713890e-763d-4a51-bffb-6cf8dd12fd18\") " pod="openshift-image-registry/node-ca-n4b5s"
Nov 04 03:16:26.880251 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.880165    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-etc-systemd\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:16:26.880251 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.880215    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-run-systemd\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.880309 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.880254    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9trx7\" (UniqueName: \"kubernetes.io/projected/21a42913-5c4e-4057-aae5-2804038f16e4-kube-api-access-9trx7\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.880309 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.880275    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d10e9268-df07-428b-ae25-528d9fe075d2-kubelet-config\") pod \"global-pull-secret-syncer-tkkhk\" (UID: \"d10e9268-df07-428b-ae25-528d9fe075d2\") " pod="kube-system/global-pull-secret-syncer-tkkhk"
Nov 04 03:16:26.880309 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.880303    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ff9f5afb-957c-4383-8fa9-698f7ada29b0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-67bjk\" (UID: \"ff9f5afb-957c-4383-8fa9-698f7ada29b0\") " pod="openshift-multus/multus-additional-cni-plugins-67bjk"
Nov 04 03:16:26.880418 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.880318    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-host-var-lib-kubelet\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:26.880418 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.880332    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-host-run-multus-certs\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:26.880418 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.880346    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-sys\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:16:26.880418 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.880359    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ff9f5afb-957c-4383-8fa9-698f7ada29b0-cnibin\") pod \"multus-additional-cni-plugins-67bjk\" (UID: \"ff9f5afb-957c-4383-8fa9-698f7ada29b0\") " pod="openshift-multus/multus-additional-cni-plugins-67bjk"
Nov 04 03:16:26.880538 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.880426    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6c8b9\" (UniqueName: \"kubernetes.io/projected/45621191-5080-4ee9-a08e-f7b2ee49fe41-kube-api-access-6c8b9\") pod \"iptables-alerter-sdhv4\" (UID: \"45621191-5080-4ee9-a08e-f7b2ee49fe41\") " pod="openshift-network-operator/iptables-alerter-sdhv4"
Nov 04 03:16:26.880538 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.880443    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-cnibin\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:26.880538 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.880459    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-etc-sysconfig\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:16:26.880538 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.880494    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d604b334-07f4-40f2-bfa9-4b26c301c84c-registration-dir\") pod \"aws-ebs-csi-driver-node-jvtr8\" (UID: \"d604b334-07f4-40f2-bfa9-4b26c301c84c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8"
Nov 04 03:16:26.880538 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.880517    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aab0bd9b-cb90-4d9b-9f46-52c6c87a34cf-tmp-dir\") pod \"node-resolver-6pwhj\" (UID: \"aab0bd9b-cb90-4d9b-9f46-52c6c87a34cf\") " pod="openshift-dns/node-resolver-6pwhj"
Nov 04 03:16:26.880538 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.880533    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7cm5\" (UniqueName: \"kubernetes.io/projected/aab0bd9b-cb90-4d9b-9f46-52c6c87a34cf-kube-api-access-b7cm5\") pod \"node-resolver-6pwhj\" (UID: \"aab0bd9b-cb90-4d9b-9f46-52c6c87a34cf\") " pod="openshift-dns/node-resolver-6pwhj"
Nov 04 03:16:26.880712 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.880574    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-host-var-lib-cni-multus\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:26.880712 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.880590    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a683bd45-33c3-43fb-9530-292d7fea6970-tmp\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:16:26.880712 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.880604    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qm4f8\" (UniqueName: \"kubernetes.io/projected/a683bd45-33c3-43fb-9530-292d7fea6970-kube-api-access-qm4f8\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:16:26.880712 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.880626    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-etc-openvswitch\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.880712 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.880647    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ff9f5afb-957c-4383-8fa9-698f7ada29b0-cni-binary-copy\") pod \"multus-additional-cni-plugins-67bjk\" (UID: \"ff9f5afb-957c-4383-8fa9-698f7ada29b0\") " pod="openshift-multus/multus-additional-cni-plugins-67bjk"
Nov 04 03:16:26.880712 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.880662    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/aab0bd9b-cb90-4d9b-9f46-52c6c87a34cf-hosts-file\") pod \"node-resolver-6pwhj\" (UID: \"aab0bd9b-cb90-4d9b-9f46-52c6c87a34cf\") " pod="openshift-dns/node-resolver-6pwhj"
Nov 04 03:16:26.880712 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.880676    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-multus-conf-dir\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:26.880712 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.880704    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-host\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:16:26.880936 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.880731    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-host-slash\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.880936 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.880748    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-host-run-netns\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.880936 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.880771    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-host-cni-bin\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.880936 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.880802    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-system-cni-dir\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:26.880936 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.880817    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-host-var-lib-cni-bin\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:26.880936 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.880849    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-etc-modprobe-d\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:16:26.880936 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.880877    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-etc-sysctl-d\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:16:26.880936 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.880899    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a683bd45-33c3-43fb-9530-292d7fea6970-etc-tuned\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:16:26.880936 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.880934    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/21a42913-5c4e-4057-aae5-2804038f16e4-ovnkube-script-lib\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.881206 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.880952    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d604b334-07f4-40f2-bfa9-4b26c301c84c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jvtr8\" (UID: \"d604b334-07f4-40f2-bfa9-4b26c301c84c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8"
Nov 04 03:16:26.881206 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.880966    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-multus-cni-dir\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:26.881206 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.880980    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6f627240-24ef-4aee-9810-51f422083ca3-cni-binary-copy\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:26.881206 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.881000    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-systemd-units\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.881206 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.881026    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d604b334-07f4-40f2-bfa9-4b26c301c84c-etc-selinux\") pod \"aws-ebs-csi-driver-node-jvtr8\" (UID: \"d604b334-07f4-40f2-bfa9-4b26c301c84c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8"
Nov 04 03:16:26.881206 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.881045    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x424d\" (UniqueName: \"kubernetes.io/projected/ff9f5afb-957c-4383-8fa9-698f7ada29b0-kube-api-access-x424d\") pod \"multus-additional-cni-plugins-67bjk\" (UID: \"ff9f5afb-957c-4383-8fa9-698f7ada29b0\") " pod="openshift-multus/multus-additional-cni-plugins-67bjk"
Nov 04 03:16:26.881206 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.881070    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-lib-modules\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:16:26.881206 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.881093    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-host-kubelet\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.881206 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.881115    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/21a42913-5c4e-4057-aae5-2804038f16e4-ovn-node-metrics-cert\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.881206 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.881134    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5krfn\" (UniqueName: \"kubernetes.io/projected/fc3d3c0e-2877-4264-93a7-be3756d2dc5c-kube-api-access-5krfn\") pod \"network-metrics-daemon-lngrr\" (UID: \"fc3d3c0e-2877-4264-93a7-be3756d2dc5c\") " pod="openshift-multus/network-metrics-daemon-lngrr"
Nov 04 03:16:26.881206 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.881167    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/45621191-5080-4ee9-a08e-f7b2ee49fe41-host-slash\") pod \"iptables-alerter-sdhv4\" (UID: \"45621191-5080-4ee9-a08e-f7b2ee49fe41\") " pod="openshift-network-operator/iptables-alerter-sdhv4"
Nov 04 03:16:26.881206 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.881190    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d10e9268-df07-428b-ae25-528d9fe075d2-dbus\") pod \"global-pull-secret-syncer-tkkhk\" (UID: \"d10e9268-df07-428b-ae25-528d9fe075d2\") " pod="kube-system/global-pull-secret-syncer-tkkhk"
Nov 04 03:16:26.881556 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.881216    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-host-run-netns\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:26.881556 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.881252    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kl5q\" (UniqueName: \"kubernetes.io/projected/6f627240-24ef-4aee-9810-51f422083ca3-kube-api-access-2kl5q\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:26.881556 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.881272    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-etc-kubernetes\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:16:26.881556 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.881288    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-run-openvswitch\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.881556 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.881304    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-log-socket\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.881556 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.881323    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.881556 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.881342    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d10e9268-df07-428b-ae25-528d9fe075d2-original-pull-secret\") pod \"global-pull-secret-syncer-tkkhk\" (UID: \"d10e9268-df07-428b-ae25-528d9fe075d2\") " pod="kube-system/global-pull-secret-syncer-tkkhk"
Nov 04 03:16:26.881556 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.881357    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-etc-sysctl-conf\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:16:26.881556 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.881371    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jltvv\" (UniqueName: \"kubernetes.io/projected/04b955ef-528b-4573-b18f-0957658f79e4-kube-api-access-jltvv\") pod \"network-check-target-hjssc\" (UID: \"04b955ef-528b-4573-b18f-0957658f79e4\") " pod="openshift-network-diagnostics/network-check-target-hjssc"
Nov 04 03:16:26.881556 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.881390    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d604b334-07f4-40f2-bfa9-4b26c301c84c-sys-fs\") pod \"aws-ebs-csi-driver-node-jvtr8\" (UID: \"d604b334-07f4-40f2-bfa9-4b26c301c84c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8"
Nov 04 03:16:26.881556 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.881433    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ff9f5afb-957c-4383-8fa9-698f7ada29b0-system-cni-dir\") pod \"multus-additional-cni-plugins-67bjk\" (UID: \"ff9f5afb-957c-4383-8fa9-698f7ada29b0\") " pod="openshift-multus/multus-additional-cni-plugins-67bjk"
Nov 04 03:16:26.881556 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.881448    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-multus-socket-dir-parent\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:26.881556 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.881464    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-host-run-k8s-cni-cncf-io\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:26.881556 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.881477    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-run\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:16:26.881556 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.881491    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d604b334-07f4-40f2-bfa9-4b26c301c84c-device-dir\") pod \"aws-ebs-csi-driver-node-jvtr8\" (UID: \"d604b334-07f4-40f2-bfa9-4b26c301c84c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8"
Nov 04 03:16:26.881556 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.881512    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hp4bb\" (UniqueName: \"kubernetes.io/projected/7713890e-763d-4a51-bffb-6cf8dd12fd18-kube-api-access-hp4bb\") pod \"node-ca-n4b5s\" (UID: \"7713890e-763d-4a51-bffb-6cf8dd12fd18\") " pod="openshift-image-registry/node-ca-n4b5s"
Nov 04 03:16:26.881980 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.881534    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6f627240-24ef-4aee-9810-51f422083ca3-multus-daemon-config\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:26.881980 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.881556    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-etc-kubernetes\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:26.881980 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.881583    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-node-log\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.881980 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.881607    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/45621191-5080-4ee9-a08e-f7b2ee49fe41-iptables-alerter-script\") pod \"iptables-alerter-sdhv4\" (UID: \"45621191-5080-4ee9-a08e-f7b2ee49fe41\") " pod="openshift-network-operator/iptables-alerter-sdhv4"
Nov 04 03:16:26.881980 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.881625    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-hostroot\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:26.881980 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.881657    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-var-lib-kubelet\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:16:26.881980 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.881679    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-run-ovn\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.881980 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.881699    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-host-cni-netd\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.881980 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.881714    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/21a42913-5c4e-4057-aae5-2804038f16e4-env-overrides\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.881980 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.881727    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d604b334-07f4-40f2-bfa9-4b26c301c84c-socket-dir\") pod \"aws-ebs-csi-driver-node-jvtr8\" (UID: \"d604b334-07f4-40f2-bfa9-4b26c301c84c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8"
Nov 04 03:16:26.881980 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.881743    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ff9f5afb-957c-4383-8fa9-698f7ada29b0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-67bjk\" (UID: \"ff9f5afb-957c-4383-8fa9-698f7ada29b0\") " pod="openshift-multus/multus-additional-cni-plugins-67bjk"
Nov 04 03:16:26.881980 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.881763    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/029e7b7b-265e-459a-bc09-a90b5c179993-agent-certs\") pod \"konnectivity-agent-m2vkr\" (UID: \"029e7b7b-265e-459a-bc09-a90b5c179993\") " pod="kube-system/konnectivity-agent-m2vkr"
Nov 04 03:16:26.881980 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.881777    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-var-lib-openvswitch\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.881980 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.881807    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ff9f5afb-957c-4383-8fa9-698f7ada29b0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-67bjk\" (UID: \"ff9f5afb-957c-4383-8fa9-698f7ada29b0\") " pod="openshift-multus/multus-additional-cni-plugins-67bjk"
Nov 04 03:16:26.881980 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.881830    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/029e7b7b-265e-459a-bc09-a90b5c179993-konnectivity-ca\") pod \"konnectivity-agent-m2vkr\" (UID: \"029e7b7b-265e-459a-bc09-a90b5c179993\") " pod="kube-system/konnectivity-agent-m2vkr"
Nov 04 03:16:26.881980 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.881852    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-host-run-ovn-kubernetes\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.882415 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.881880    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/21a42913-5c4e-4057-aae5-2804038f16e4-ovnkube-config\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.888383 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.888357    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 04 03:16:26.888493 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.888415    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 04 03:16:26.888608 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.888590    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-9cxzk\""
Nov 04 03:16:26.888666 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.888651    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 04 03:16:26.888718 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.888671    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 04 03:16:26.888771 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.888746    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 04 03:16:26.889200 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.889186    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-fbcnm\""
Nov 04 03:16:26.890958 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.890939    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 04 03:16:26.891070 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.891005    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 04 03:16:26.891159 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.891142    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-569bn\""
Nov 04 03:16:26.891217 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.891172    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 04 03:16:26.891217 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.891189    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 04 03:16:26.891542 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.891498    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 04 03:16:26.891542 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.891507    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-x6xxb\""
Nov 04 03:16:26.891733 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.891566    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 04 03:16:26.891733 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.891572    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 04 03:16:26.891733 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.891610    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 04 03:16:26.891733 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.891629    2579 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 04 03:16:26.891733 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.891667    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-bfx4z"
Nov 04 03:16:26.891733 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.891679    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-4vrb5\""
Nov 04 03:16:26.892051 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.892037    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 04 03:16:26.983043 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.983004    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5cdl\" (UniqueName: \"kubernetes.io/projected/d604b334-07f4-40f2-bfa9-4b26c301c84c-kube-api-access-n5cdl\") pod \"aws-ebs-csi-driver-node-jvtr8\" (UID: \"d604b334-07f4-40f2-bfa9-4b26c301c84c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8"
Nov 04 03:16:26.983043 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.983040    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc3d3c0e-2877-4264-93a7-be3756d2dc5c-metrics-certs\") pod \"network-metrics-daemon-lngrr\" (UID: \"fc3d3c0e-2877-4264-93a7-be3756d2dc5c\") " pod="openshift-multus/network-metrics-daemon-lngrr"
Nov 04 03:16:26.983263 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.983057    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7713890e-763d-4a51-bffb-6cf8dd12fd18-host\") pod \"node-ca-n4b5s\" (UID: \"7713890e-763d-4a51-bffb-6cf8dd12fd18\") " pod="openshift-image-registry/node-ca-n4b5s"
Nov 04 03:16:26.983263 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.983071    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7713890e-763d-4a51-bffb-6cf8dd12fd18-serviceca\") pod \"node-ca-n4b5s\" (UID: \"7713890e-763d-4a51-bffb-6cf8dd12fd18\") " pod="openshift-image-registry/node-ca-n4b5s"
Nov 04 03:16:26.983263 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.983086    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-etc-systemd\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:16:26.983263 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.983099    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-run-systemd\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.983263 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.983128    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7713890e-763d-4a51-bffb-6cf8dd12fd18-host\") pod \"node-ca-n4b5s\" (UID: \"7713890e-763d-4a51-bffb-6cf8dd12fd18\") " pod="openshift-image-registry/node-ca-n4b5s"
Nov 04 03:16:26.983493 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.983205    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-run-systemd\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.983541 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.983475    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9trx7\" (UniqueName: \"kubernetes.io/projected/21a42913-5c4e-4057-aae5-2804038f16e4-kube-api-access-9trx7\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.983541 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.983534    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d10e9268-df07-428b-ae25-528d9fe075d2-kubelet-config\") pod \"global-pull-secret-syncer-tkkhk\" (UID: \"d10e9268-df07-428b-ae25-528d9fe075d2\") " pod="kube-system/global-pull-secret-syncer-tkkhk"
Nov 04 03:16:26.983629 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:26.983547    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:16:26.983629 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.983559    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ff9f5afb-957c-4383-8fa9-698f7ada29b0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-67bjk\" (UID: \"ff9f5afb-957c-4383-8fa9-698f7ada29b0\") " pod="openshift-multus/multus-additional-cni-plugins-67bjk"
Nov 04 03:16:26.983713 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.983629    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-host-var-lib-kubelet\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:26.983713 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.983484    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-etc-systemd\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:16:26.983713 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:26.983668    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc3d3c0e-2877-4264-93a7-be3756d2dc5c-metrics-certs podName:fc3d3c0e-2877-4264-93a7-be3756d2dc5c nodeName:}" failed. No retries permitted until 2026-11-04 03:16:27.483620349 +0000 UTC m=+2.312592882 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fc3d3c0e-2877-4264-93a7-be3756d2dc5c-metrics-certs") pod "network-metrics-daemon-lngrr" (UID: "fc3d3c0e-2877-4264-93a7-be3756d2dc5c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:16:26.983713 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.983682    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-host-var-lib-kubelet\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:26.983923 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.983827    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d10e9268-df07-428b-ae25-528d9fe075d2-kubelet-config\") pod \"global-pull-secret-syncer-tkkhk\" (UID: \"d10e9268-df07-428b-ae25-528d9fe075d2\") " pod="kube-system/global-pull-secret-syncer-tkkhk"
Nov 04 03:16:26.983923 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.983830    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ff9f5afb-957c-4383-8fa9-698f7ada29b0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-67bjk\" (UID: \"ff9f5afb-957c-4383-8fa9-698f7ada29b0\") " pod="openshift-multus/multus-additional-cni-plugins-67bjk"
Nov 04 03:16:26.984019 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.983970    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-host-run-multus-certs\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:26.984019 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.983973    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7713890e-763d-4a51-bffb-6cf8dd12fd18-serviceca\") pod \"node-ca-n4b5s\" (UID: \"7713890e-763d-4a51-bffb-6cf8dd12fd18\") " pod="openshift-image-registry/node-ca-n4b5s"
Nov 04 03:16:26.984019 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.984013    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-host-run-multus-certs\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:26.984156 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.984059    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-sys\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:16:26.984156 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.984094    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ff9f5afb-957c-4383-8fa9-698f7ada29b0-cnibin\") pod \"multus-additional-cni-plugins-67bjk\" (UID: \"ff9f5afb-957c-4383-8fa9-698f7ada29b0\") " pod="openshift-multus/multus-additional-cni-plugins-67bjk"
Nov 04 03:16:26.984156 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.984114    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-sys\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:16:26.984156 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.984125    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6c8b9\" (UniqueName: \"kubernetes.io/projected/45621191-5080-4ee9-a08e-f7b2ee49fe41-kube-api-access-6c8b9\") pod \"iptables-alerter-sdhv4\" (UID: \"45621191-5080-4ee9-a08e-f7b2ee49fe41\") " pod="openshift-network-operator/iptables-alerter-sdhv4"
Nov 04 03:16:26.984156 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.984126    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ff9f5afb-957c-4383-8fa9-698f7ada29b0-cnibin\") pod \"multus-additional-cni-plugins-67bjk\" (UID: \"ff9f5afb-957c-4383-8fa9-698f7ada29b0\") " pod="openshift-multus/multus-additional-cni-plugins-67bjk"
Nov 04 03:16:26.984414 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.984167    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-cnibin\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:26.984414 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.984202    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-etc-sysconfig\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:16:26.984414 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.984236    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d604b334-07f4-40f2-bfa9-4b26c301c84c-registration-dir\") pod \"aws-ebs-csi-driver-node-jvtr8\" (UID: \"d604b334-07f4-40f2-bfa9-4b26c301c84c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8"
Nov 04 03:16:26.984414 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.984269    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aab0bd9b-cb90-4d9b-9f46-52c6c87a34cf-tmp-dir\") pod \"node-resolver-6pwhj\" (UID: \"aab0bd9b-cb90-4d9b-9f46-52c6c87a34cf\") " pod="openshift-dns/node-resolver-6pwhj"
Nov 04 03:16:26.984414 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.984296    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7cm5\" (UniqueName: \"kubernetes.io/projected/aab0bd9b-cb90-4d9b-9f46-52c6c87a34cf-kube-api-access-b7cm5\") pod \"node-resolver-6pwhj\" (UID: \"aab0bd9b-cb90-4d9b-9f46-52c6c87a34cf\") " pod="openshift-dns/node-resolver-6pwhj"
Nov 04 03:16:26.984414 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.984330    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-host-var-lib-cni-multus\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:26.984414 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.984361    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a683bd45-33c3-43fb-9530-292d7fea6970-tmp\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:16:26.984414 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.984372    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d604b334-07f4-40f2-bfa9-4b26c301c84c-registration-dir\") pod \"aws-ebs-csi-driver-node-jvtr8\" (UID: \"d604b334-07f4-40f2-bfa9-4b26c301c84c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8"
Nov 04 03:16:26.984414 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.984393    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qm4f8\" (UniqueName: \"kubernetes.io/projected/a683bd45-33c3-43fb-9530-292d7fea6970-kube-api-access-qm4f8\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:16:26.984847 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.984441    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-etc-openvswitch\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.984847 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.984455    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-etc-sysconfig\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:16:26.984847 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.984469    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ff9f5afb-957c-4383-8fa9-698f7ada29b0-cni-binary-copy\") pod \"multus-additional-cni-plugins-67bjk\" (UID: \"ff9f5afb-957c-4383-8fa9-698f7ada29b0\") " pod="openshift-multus/multus-additional-cni-plugins-67bjk"
Nov 04 03:16:26.984847 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.984505    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/aab0bd9b-cb90-4d9b-9f46-52c6c87a34cf-hosts-file\") pod \"node-resolver-6pwhj\" (UID: \"aab0bd9b-cb90-4d9b-9f46-52c6c87a34cf\") " pod="openshift-dns/node-resolver-6pwhj"
Nov 04 03:16:26.984847 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.984509    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-cnibin\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:26.984847 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.984538    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-multus-conf-dir\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:26.984847 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.984568    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-host\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:16:26.984847 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.984600    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-host-slash\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.984847 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.984628    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-host-run-netns\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.984847 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.984667    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-host-cni-bin\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.984847 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.984701    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-system-cni-dir\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:26.984847 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.984735    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-host-var-lib-cni-bin\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:26.984847 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.984775    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-etc-modprobe-d\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:16:26.984847 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.984802    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-etc-sysctl-d\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:16:26.984847 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.984801    2579 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 04 03:16:26.984847 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.984836    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a683bd45-33c3-43fb-9530-292d7fea6970-etc-tuned\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:16:26.985682 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.985301    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/21a42913-5c4e-4057-aae5-2804038f16e4-ovnkube-script-lib\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.985682 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.985341    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d604b334-07f4-40f2-bfa9-4b26c301c84c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jvtr8\" (UID: \"d604b334-07f4-40f2-bfa9-4b26c301c84c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8"
Nov 04 03:16:26.985682 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.985372    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-multus-cni-dir\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:26.985682 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.985414    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6f627240-24ef-4aee-9810-51f422083ca3-cni-binary-copy\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:26.985682 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.985446    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-systemd-units\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.985682 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.985479    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d604b334-07f4-40f2-bfa9-4b26c301c84c-etc-selinux\") pod \"aws-ebs-csi-driver-node-jvtr8\" (UID: \"d604b334-07f4-40f2-bfa9-4b26c301c84c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8"
Nov 04 03:16:26.985682 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.985512    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x424d\" (UniqueName: \"kubernetes.io/projected/ff9f5afb-957c-4383-8fa9-698f7ada29b0-kube-api-access-x424d\") pod \"multus-additional-cni-plugins-67bjk\" (UID: \"ff9f5afb-957c-4383-8fa9-698f7ada29b0\") " pod="openshift-multus/multus-additional-cni-plugins-67bjk"
Nov 04 03:16:26.985682 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.985546    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-lib-modules\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:16:26.985682 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.985580    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-host-kubelet\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.985682 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.985612    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/21a42913-5c4e-4057-aae5-2804038f16e4-ovn-node-metrics-cert\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.985682 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.985643    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5krfn\" (UniqueName: \"kubernetes.io/projected/fc3d3c0e-2877-4264-93a7-be3756d2dc5c-kube-api-access-5krfn\") pod \"network-metrics-daemon-lngrr\" (UID: \"fc3d3c0e-2877-4264-93a7-be3756d2dc5c\") " pod="openshift-multus/network-metrics-daemon-lngrr"
Nov 04 03:16:26.985682 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.985674    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/45621191-5080-4ee9-a08e-f7b2ee49fe41-host-slash\") pod \"iptables-alerter-sdhv4\" (UID: \"45621191-5080-4ee9-a08e-f7b2ee49fe41\") " pod="openshift-network-operator/iptables-alerter-sdhv4"
Nov 04 03:16:26.986262 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.985707    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d10e9268-df07-428b-ae25-528d9fe075d2-dbus\") pod \"global-pull-secret-syncer-tkkhk\" (UID: \"d10e9268-df07-428b-ae25-528d9fe075d2\") " pod="kube-system/global-pull-secret-syncer-tkkhk"
Nov 04 03:16:26.986262 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.985754    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-host-run-netns\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:26.986262 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.985781    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kl5q\" (UniqueName: \"kubernetes.io/projected/6f627240-24ef-4aee-9810-51f422083ca3-kube-api-access-2kl5q\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:26.986262 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.985813    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-etc-kubernetes\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:16:26.986262 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.985847    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-run-openvswitch\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.986262 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.985880    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-log-socket\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.986262 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.985912    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.986262 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.985943    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d10e9268-df07-428b-ae25-528d9fe075d2-original-pull-secret\") pod \"global-pull-secret-syncer-tkkhk\" (UID: \"d10e9268-df07-428b-ae25-528d9fe075d2\") " pod="kube-system/global-pull-secret-syncer-tkkhk"
Nov 04 03:16:26.986262 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.985980    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-etc-sysctl-conf\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:16:26.986262 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.986014    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jltvv\" (UniqueName: \"kubernetes.io/projected/04b955ef-528b-4573-b18f-0957658f79e4-kube-api-access-jltvv\") pod \"network-check-target-hjssc\" (UID: \"04b955ef-528b-4573-b18f-0957658f79e4\") " pod="openshift-network-diagnostics/network-check-target-hjssc"
Nov 04 03:16:26.986262 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.986047    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d604b334-07f4-40f2-bfa9-4b26c301c84c-sys-fs\") pod \"aws-ebs-csi-driver-node-jvtr8\" (UID: \"d604b334-07f4-40f2-bfa9-4b26c301c84c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8"
Nov 04 03:16:26.986262 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.986074    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ff9f5afb-957c-4383-8fa9-698f7ada29b0-system-cni-dir\") pod \"multus-additional-cni-plugins-67bjk\" (UID: \"ff9f5afb-957c-4383-8fa9-698f7ada29b0\") " pod="openshift-multus/multus-additional-cni-plugins-67bjk"
Nov 04 03:16:26.986262 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.986107    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-multus-socket-dir-parent\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:26.986262 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.986137    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-host-run-k8s-cni-cncf-io\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:26.986262 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.986169    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-run\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:16:26.986262 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.986199    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d604b334-07f4-40f2-bfa9-4b26c301c84c-device-dir\") pod \"aws-ebs-csi-driver-node-jvtr8\" (UID: \"d604b334-07f4-40f2-bfa9-4b26c301c84c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8"
Nov 04 03:16:26.986262 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.986226    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hp4bb\" (UniqueName: \"kubernetes.io/projected/7713890e-763d-4a51-bffb-6cf8dd12fd18-kube-api-access-hp4bb\") pod \"node-ca-n4b5s\" (UID: \"7713890e-763d-4a51-bffb-6cf8dd12fd18\") " pod="openshift-image-registry/node-ca-n4b5s"
Nov 04 03:16:26.987108 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.986259    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6f627240-24ef-4aee-9810-51f422083ca3-multus-daemon-config\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:26.987108 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.986289    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-etc-kubernetes\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:26.987108 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.986322    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-node-log\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.987108 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.986354    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/45621191-5080-4ee9-a08e-f7b2ee49fe41-iptables-alerter-script\") pod \"iptables-alerter-sdhv4\" (UID: \"45621191-5080-4ee9-a08e-f7b2ee49fe41\") " pod="openshift-network-operator/iptables-alerter-sdhv4"
Nov 04 03:16:26.987108 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.986379    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-hostroot\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:26.987108 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.986428    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-var-lib-kubelet\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:16:26.987108 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.986551    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-multus-conf-dir\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:26.987108 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.986559    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ff9f5afb-957c-4383-8fa9-698f7ada29b0-system-cni-dir\") pod \"multus-additional-cni-plugins-67bjk\" (UID: \"ff9f5afb-957c-4383-8fa9-698f7ada29b0\") " pod="openshift-multus/multus-additional-cni-plugins-67bjk"
Nov 04 03:16:26.987108 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.986559    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d604b334-07f4-40f2-bfa9-4b26c301c84c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jvtr8\" (UID: \"d604b334-07f4-40f2-bfa9-4b26c301c84c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8"
Nov 04 03:16:26.987108 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.986589    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-run-ovn\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.987108 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.986620    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-etc-modprobe-d\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:16:26.987108 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.986630    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-host-cni-netd\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.987108 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.986635    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-host-kubelet\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.987108 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.986645    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-host-run-netns\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.987108 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.986653    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.987108 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.986663    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/21a42913-5c4e-4057-aae5-2804038f16e4-env-overrides\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.987108 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.986699    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d604b334-07f4-40f2-bfa9-4b26c301c84c-socket-dir\") pod \"aws-ebs-csi-driver-node-jvtr8\" (UID: \"d604b334-07f4-40f2-bfa9-4b26c301c84c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8"
Nov 04 03:16:26.987929 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.986703    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-host-var-lib-cni-multus\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:26.987929 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.986704    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-host-slash\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.987929 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.986734    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ff9f5afb-957c-4383-8fa9-698f7ada29b0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-67bjk\" (UID: \"ff9f5afb-957c-4383-8fa9-698f7ada29b0\") " pod="openshift-multus/multus-additional-cni-plugins-67bjk"
Nov 04 03:16:26.987929 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.986757    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-etc-sysctl-d\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:16:26.987929 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.986663    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-var-lib-kubelet\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:16:26.987929 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.986777    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-run-openvswitch\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.987929 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.986829    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d604b334-07f4-40f2-bfa9-4b26c301c84c-socket-dir\") pod \"aws-ebs-csi-driver-node-jvtr8\" (UID: \"d604b334-07f4-40f2-bfa9-4b26c301c84c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8"
Nov 04 03:16:26.987929 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.986833    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-host-cni-bin\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.987929 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.986771    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/029e7b7b-265e-459a-bc09-a90b5c179993-agent-certs\") pod \"konnectivity-agent-m2vkr\" (UID: \"029e7b7b-265e-459a-bc09-a90b5c179993\") " pod="kube-system/konnectivity-agent-m2vkr"
Nov 04 03:16:26.987929 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.986885    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-log-socket\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.987929 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.986885    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-var-lib-openvswitch\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.987929 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.986910    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-var-lib-openvswitch\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.987929 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.986928    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ff9f5afb-957c-4383-8fa9-698f7ada29b0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-67bjk\" (UID: \"ff9f5afb-957c-4383-8fa9-698f7ada29b0\") " pod="openshift-multus/multus-additional-cni-plugins-67bjk"
Nov 04 03:16:26.987929 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.986951    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aab0bd9b-cb90-4d9b-9f46-52c6c87a34cf-tmp-dir\") pod \"node-resolver-6pwhj\" (UID: \"aab0bd9b-cb90-4d9b-9f46-52c6c87a34cf\") " pod="openshift-dns/node-resolver-6pwhj"
Nov 04 03:16:26.987929 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.986962    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/029e7b7b-265e-459a-bc09-a90b5c179993-konnectivity-ca\") pod \"konnectivity-agent-m2vkr\" (UID: \"029e7b7b-265e-459a-bc09-a90b5c179993\") " pod="kube-system/konnectivity-agent-m2vkr"
Nov 04 03:16:26.987929 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.987215    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/21a42913-5c4e-4057-aae5-2804038f16e4-env-overrides\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.987929 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.987369    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ff9f5afb-957c-4383-8fa9-698f7ada29b0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-67bjk\" (UID: \"ff9f5afb-957c-4383-8fa9-698f7ada29b0\") " pod="openshift-multus/multus-additional-cni-plugins-67bjk"
Nov 04 03:16:26.988783 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.986699    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-host-cni-netd\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.988783 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.987486    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-etc-openvswitch\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.988783 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.987501    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d604b334-07f4-40f2-bfa9-4b26c301c84c-sys-fs\") pod \"aws-ebs-csi-driver-node-jvtr8\" (UID: \"d604b334-07f4-40f2-bfa9-4b26c301c84c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8"
Nov 04 03:16:26.988783 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.987530    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/45621191-5080-4ee9-a08e-f7b2ee49fe41-host-slash\") pod \"iptables-alerter-sdhv4\" (UID: \"45621191-5080-4ee9-a08e-f7b2ee49fe41\") " pod="openshift-network-operator/iptables-alerter-sdhv4"
Nov 04 03:16:26.988783 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.987858    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-host-run-k8s-cni-cncf-io\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:26.988783 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.987631    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-host\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:16:26.988783 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.987673    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-multus-cni-dir\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:26.988783 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.987752    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/21a42913-5c4e-4057-aae5-2804038f16e4-ovnkube-script-lib\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.988783 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.987821    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-multus-socket-dir-parent\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:26.988783 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.987919    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-run-ovn\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.988783 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.987944    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ff9f5afb-957c-4383-8fa9-698f7ada29b0-cni-binary-copy\") pod \"multus-additional-cni-plugins-67bjk\" (UID: \"ff9f5afb-957c-4383-8fa9-698f7ada29b0\") " pod="openshift-multus/multus-additional-cni-plugins-67bjk"
Nov 04 03:16:26.988783 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.988007    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d604b334-07f4-40f2-bfa9-4b26c301c84c-etc-selinux\") pod \"aws-ebs-csi-driver-node-jvtr8\" (UID: \"d604b334-07f4-40f2-bfa9-4b26c301c84c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8"
Nov 04 03:16:26.988783 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.988098    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-etc-kubernetes\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:16:26.988783 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.988157    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-etc-kubernetes\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:26.988783 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.987571    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-host-var-lib-cni-bin\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:26.988783 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.988259    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-system-cni-dir\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:26.988783 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.988324    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-lib-modules\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:16:26.988783 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.988349    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6f627240-24ef-4aee-9810-51f422083ca3-cni-binary-copy\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:26.989622 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.988597    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-host-run-ovn-kubernetes\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.989622 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.988641    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/21a42913-5c4e-4057-aae5-2804038f16e4-ovnkube-config\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.989622 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.988706    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-etc-sysctl-conf\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:16:26.989622 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.988773    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6f627240-24ef-4aee-9810-51f422083ca3-multus-daemon-config\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:26.989622 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.988774    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-host-run-ovn-kubernetes\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.989622 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.988846    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-node-log\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.989622 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.989351    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-run\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:16:26.989622 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:26.989355    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 03:16:26.989622 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.989393    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-systemd-units\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.989622 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:26.989468    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d10e9268-df07-428b-ae25-528d9fe075d2-original-pull-secret podName:d10e9268-df07-428b-ae25-528d9fe075d2 nodeName:}" failed. No retries permitted until 2026-11-04 03:16:27.489424996 +0000 UTC m=+2.318397525 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d10e9268-df07-428b-ae25-528d9fe075d2-original-pull-secret") pod "global-pull-secret-syncer-tkkhk" (UID: "d10e9268-df07-428b-ae25-528d9fe075d2") : object "kube-system"/"original-pull-secret" not registered
Nov 04 03:16:26.989622 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.989484    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/aab0bd9b-cb90-4d9b-9f46-52c6c87a34cf-hosts-file\") pod \"node-resolver-6pwhj\" (UID: \"aab0bd9b-cb90-4d9b-9f46-52c6c87a34cf\") " pod="openshift-dns/node-resolver-6pwhj"
Nov 04 03:16:26.989622 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.989493    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d10e9268-df07-428b-ae25-528d9fe075d2-dbus\") pod \"global-pull-secret-syncer-tkkhk\" (UID: \"d10e9268-df07-428b-ae25-528d9fe075d2\") " pod="kube-system/global-pull-secret-syncer-tkkhk"
Nov 04 03:16:26.989622 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.989547    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-host-run-netns\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:26.989622 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.989552    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d604b334-07f4-40f2-bfa9-4b26c301c84c-device-dir\") pod \"aws-ebs-csi-driver-node-jvtr8\" (UID: \"d604b334-07f4-40f2-bfa9-4b26c301c84c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8"
Nov 04 03:16:26.990345 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.989648    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-hostroot\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:26.990345 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.990261    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/45621191-5080-4ee9-a08e-f7b2ee49fe41-iptables-alerter-script\") pod \"iptables-alerter-sdhv4\" (UID: \"45621191-5080-4ee9-a08e-f7b2ee49fe41\") " pod="openshift-network-operator/iptables-alerter-sdhv4"
Nov 04 03:16:26.990623 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.990604    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/029e7b7b-265e-459a-bc09-a90b5c179993-agent-certs\") pod \"konnectivity-agent-m2vkr\" (UID: \"029e7b7b-265e-459a-bc09-a90b5c179993\") " pod="kube-system/konnectivity-agent-m2vkr"
Nov 04 03:16:26.990806 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.990790    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/21a42913-5c4e-4057-aae5-2804038f16e4-ovn-node-metrics-cert\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.990867 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.990834    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ff9f5afb-957c-4383-8fa9-698f7ada29b0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-67bjk\" (UID: \"ff9f5afb-957c-4383-8fa9-698f7ada29b0\") " pod="openshift-multus/multus-additional-cni-plugins-67bjk"
Nov 04 03:16:26.990982 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.990962    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a683bd45-33c3-43fb-9530-292d7fea6970-tmp\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:16:26.991033 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.990989    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a683bd45-33c3-43fb-9530-292d7fea6970-etc-tuned\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:16:26.991085 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.991069    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/21a42913-5c4e-4057-aae5-2804038f16e4-ovnkube-config\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:26.991133 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:26.991103    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/029e7b7b-265e-459a-bc09-a90b5c179993-konnectivity-ca\") pod \"konnectivity-agent-m2vkr\" (UID: \"029e7b7b-265e-459a-bc09-a90b5c179993\") " pod="kube-system/konnectivity-agent-m2vkr"
Nov 04 03:16:27.043754 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:27.043722    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6c8b9\" (UniqueName: \"kubernetes.io/projected/45621191-5080-4ee9-a08e-f7b2ee49fe41-kube-api-access-6c8b9\") pod \"iptables-alerter-sdhv4\" (UID: \"45621191-5080-4ee9-a08e-f7b2ee49fe41\") " pod="openshift-network-operator/iptables-alerter-sdhv4"
Nov 04 03:16:27.043917 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:27.043722    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9trx7\" (UniqueName: \"kubernetes.io/projected/21a42913-5c4e-4057-aae5-2804038f16e4-kube-api-access-9trx7\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:27.048885 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:27.048850    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hp4bb\" (UniqueName: \"kubernetes.io/projected/7713890e-763d-4a51-bffb-6cf8dd12fd18-kube-api-access-hp4bb\") pod \"node-ca-n4b5s\" (UID: \"7713890e-763d-4a51-bffb-6cf8dd12fd18\") " pod="openshift-image-registry/node-ca-n4b5s"
Nov 04 03:16:27.049021 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:27.048854    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7cm5\" (UniqueName: \"kubernetes.io/projected/aab0bd9b-cb90-4d9b-9f46-52c6c87a34cf-kube-api-access-b7cm5\") pod \"node-resolver-6pwhj\" (UID: \"aab0bd9b-cb90-4d9b-9f46-52c6c87a34cf\") " pod="openshift-dns/node-resolver-6pwhj"
Nov 04 03:16:27.049021 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:27.048999    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qm4f8\" (UniqueName: \"kubernetes.io/projected/a683bd45-33c3-43fb-9530-292d7fea6970-kube-api-access-qm4f8\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:16:27.049137 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:27.049080    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5cdl\" (UniqueName: \"kubernetes.io/projected/d604b334-07f4-40f2-bfa9-4b26c301c84c-kube-api-access-n5cdl\") pod \"aws-ebs-csi-driver-node-jvtr8\" (UID: \"d604b334-07f4-40f2-bfa9-4b26c301c84c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8"
Nov 04 03:16:27.049750 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:27.049702    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x424d\" (UniqueName: \"kubernetes.io/projected/ff9f5afb-957c-4383-8fa9-698f7ada29b0-kube-api-access-x424d\") pod \"multus-additional-cni-plugins-67bjk\" (UID: \"ff9f5afb-957c-4383-8fa9-698f7ada29b0\") " pod="openshift-multus/multus-additional-cni-plugins-67bjk"
Nov 04 03:16:27.050023 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:27.050008    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5krfn\" (UniqueName: \"kubernetes.io/projected/fc3d3c0e-2877-4264-93a7-be3756d2dc5c-kube-api-access-5krfn\") pod \"network-metrics-daemon-lngrr\" (UID: \"fc3d3c0e-2877-4264-93a7-be3756d2dc5c\") " pod="openshift-multus/network-metrics-daemon-lngrr"
Nov 04 03:16:27.050099 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:27.050079    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kl5q\" (UniqueName: \"kubernetes.io/projected/6f627240-24ef-4aee-9810-51f422083ca3-kube-api-access-2kl5q\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:27.050374 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:27.050360    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 03:16:27.050421 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:27.050377    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 03:16:27.050421 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:27.050387    2579 projected.go:196] Error preparing data for projected volume kube-api-access-jltvv for pod openshift-network-diagnostics/network-check-target-hjssc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:16:27.050482 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:27.050451    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/04b955ef-528b-4573-b18f-0957658f79e4-kube-api-access-jltvv podName:04b955ef-528b-4573-b18f-0957658f79e4 nodeName:}" failed. No retries permitted until 2026-11-04 03:16:27.55043856 +0000 UTC m=+2.379411089 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-jltvv" (UniqueName: "kubernetes.io/projected/04b955ef-528b-4573-b18f-0957658f79e4-kube-api-access-jltvv") pod "network-check-target-hjssc" (UID: "04b955ef-528b-4573-b18f-0957658f79e4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:16:27.142301 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:27.142266    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6pwhj"
Nov 04 03:16:27.157468 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:27.157443    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-67bjk"
Nov 04 03:16:27.168370 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:27.168342    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-m2vkr"
Nov 04 03:16:27.180622 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:27.180600    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-n4b5s"
Nov 04 03:16:27.199740 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:27.199704    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:27.200746 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:27.200724    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8"
Nov 04 03:16:27.209203 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:27.209181    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-sdhv4"
Nov 04 03:16:27.223365 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:27.223339    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9xgrv"
Nov 04 03:16:27.228542 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:27.228522    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:16:27.289053 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:27.289017    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaeb0f7512125881a2a5e1813eb42a8f7.slice/crio-5e0193cee68746668d04cd45c7dbfb28ff8d5a48f555b3213f1549e560278900 WatchSource:0}: Error finding container 5e0193cee68746668d04cd45c7dbfb28ff8d5a48f555b3213f1549e560278900: Status 404 returned error can't find the container with id 5e0193cee68746668d04cd45c7dbfb28ff8d5a48f555b3213f1549e560278900
Nov 04 03:16:27.289635 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:27.289617    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f2b595dbb5c94185c06c3dd3c0c8dce.slice/crio-18e250ae36ea4dcd84c44211ccde315ef072c36d75a4208979550f27a5d728f2 WatchSource:0}: Error finding container 18e250ae36ea4dcd84c44211ccde315ef072c36d75a4208979550f27a5d728f2: Status 404 returned error can't find the container with id 18e250ae36ea4dcd84c44211ccde315ef072c36d75a4208979550f27a5d728f2
Nov 04 03:16:27.295927 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:27.295909    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 03:16:27.378164 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:27.378103    2579 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 03:16:27.492118 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:27.492083    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc3d3c0e-2877-4264-93a7-be3756d2dc5c-metrics-certs\") pod \"network-metrics-daemon-lngrr\" (UID: \"fc3d3c0e-2877-4264-93a7-be3756d2dc5c\") " pod="openshift-multus/network-metrics-daemon-lngrr"
Nov 04 03:16:27.492278 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:27.492134    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d10e9268-df07-428b-ae25-528d9fe075d2-original-pull-secret\") pod \"global-pull-secret-syncer-tkkhk\" (UID: \"d10e9268-df07-428b-ae25-528d9fe075d2\") " pod="kube-system/global-pull-secret-syncer-tkkhk"
Nov 04 03:16:27.492278 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:27.492231    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:16:27.492278 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:27.492240    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 03:16:27.492376 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:27.492292    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc3d3c0e-2877-4264-93a7-be3756d2dc5c-metrics-certs podName:fc3d3c0e-2877-4264-93a7-be3756d2dc5c nodeName:}" failed. No retries permitted until 2026-11-04 03:16:28.492278765 +0000 UTC m=+3.321251275 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fc3d3c0e-2877-4264-93a7-be3756d2dc5c-metrics-certs") pod "network-metrics-daemon-lngrr" (UID: "fc3d3c0e-2877-4264-93a7-be3756d2dc5c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:16:27.492376 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:27.492308    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d10e9268-df07-428b-ae25-528d9fe075d2-original-pull-secret podName:d10e9268-df07-428b-ae25-528d9fe075d2 nodeName:}" failed. No retries permitted until 2026-11-04 03:16:28.492301478 +0000 UTC m=+3.321274028 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d10e9268-df07-428b-ae25-528d9fe075d2-original-pull-secret") pod "global-pull-secret-syncer-tkkhk" (UID: "d10e9268-df07-428b-ae25-528d9fe075d2") : object "kube-system"/"original-pull-secret" not registered
Nov 04 03:16:27.593450 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:27.593379    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jltvv\" (UniqueName: \"kubernetes.io/projected/04b955ef-528b-4573-b18f-0957658f79e4-kube-api-access-jltvv\") pod \"network-check-target-hjssc\" (UID: \"04b955ef-528b-4573-b18f-0957658f79e4\") " pod="openshift-network-diagnostics/network-check-target-hjssc"
Nov 04 03:16:27.593971 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:27.593538    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 03:16:27.593971 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:27.593555    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 03:16:27.593971 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:27.593563    2579 projected.go:196] Error preparing data for projected volume kube-api-access-jltvv for pod openshift-network-diagnostics/network-check-target-hjssc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:16:27.593971 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:27.593611    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/04b955ef-528b-4573-b18f-0957658f79e4-kube-api-access-jltvv podName:04b955ef-528b-4573-b18f-0957658f79e4 nodeName:}" failed. No retries permitted until 2026-11-04 03:16:28.593597827 +0000 UTC m=+3.422570354 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-jltvv" (UniqueName: "kubernetes.io/projected/04b955ef-528b-4573-b18f-0957658f79e4-kube-api-access-jltvv") pod "network-check-target-hjssc" (UID: "04b955ef-528b-4573-b18f-0957658f79e4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:16:27.740138 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:27.740044    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.us-west-2.compute.internal" event={"ID":"2f2b595dbb5c94185c06c3dd3c0c8dce","Type":"ContainerStarted","Data":"18e250ae36ea4dcd84c44211ccde315ef072c36d75a4208979550f27a5d728f2"}
Nov 04 03:16:27.741061 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:27.741036    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-50.us-west-2.compute.internal" event={"ID":"aeb0f7512125881a2a5e1813eb42a8f7","Type":"ContainerStarted","Data":"5e0193cee68746668d04cd45c7dbfb28ff8d5a48f555b3213f1549e560278900"}
Nov 04 03:16:27.893754 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:27.893723    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 03:11:26 +0000 UTC" deadline="2028-05-29 05:57:33.606869942 +0000 UTC"
Nov 04 03:16:27.893754 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:27.893748    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13730h41m5.7131243s"
Nov 04 03:16:28.086221 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:28.086199    2579 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 03:16:28.089207 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:28.089182    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaab0bd9b_cb90_4d9b_9f46_52c6c87a34cf.slice/crio-f8de65ace441d748bf714ac3874c1d9d5bdca604c8b8c76dba6c256db3d1438d WatchSource:0}: Error finding container f8de65ace441d748bf714ac3874c1d9d5bdca604c8b8c76dba6c256db3d1438d: Status 404 returned error can't find the container with id f8de65ace441d748bf714ac3874c1d9d5bdca604c8b8c76dba6c256db3d1438d
Nov 04 03:16:28.098382 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:28.098298    2579 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 03:16:28.138884 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:28.138854    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45621191_5080_4ee9_a08e_f7b2ee49fe41.slice/crio-f1f81bac5a3c3e1fc69b0c3c653eb547e6c75e948f2c3a1c3b708b5b3139249e WatchSource:0}: Error finding container f1f81bac5a3c3e1fc69b0c3c653eb547e6c75e948f2c3a1c3b708b5b3139249e: Status 404 returned error can't find the container with id f1f81bac5a3c3e1fc69b0c3c653eb547e6c75e948f2c3a1c3b708b5b3139249e
Nov 04 03:16:28.139569 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:28.139543    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd604b334_07f4_40f2_bfa9_4b26c301c84c.slice/crio-460c8b9aec9c460ded874aaca5a63be78eae1fd226e700ffc7f1587fe0828bbd WatchSource:0}: Error finding container 460c8b9aec9c460ded874aaca5a63be78eae1fd226e700ffc7f1587fe0828bbd: Status 404 returned error can't find the container with id 460c8b9aec9c460ded874aaca5a63be78eae1fd226e700ffc7f1587fe0828bbd
Nov 04 03:16:28.173200 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:28.173175    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod029e7b7b_265e_459a_bc09_a90b5c179993.slice/crio-d732058c584482205f0cda0ced5e810f97ab9a507e40b32658593463a317cfb1 WatchSource:0}: Error finding container d732058c584482205f0cda0ced5e810f97ab9a507e40b32658593463a317cfb1: Status 404 returned error can't find the container with id d732058c584482205f0cda0ced5e810f97ab9a507e40b32658593463a317cfb1
Nov 04 03:16:28.196803 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:28.196780    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff9f5afb_957c_4383_8fa9_698f7ada29b0.slice/crio-300f657b0a23c4828d40ac017087f089be7591f69de01ae10b35ad71c174dded WatchSource:0}: Error finding container 300f657b0a23c4828d40ac017087f089be7591f69de01ae10b35ad71c174dded: Status 404 returned error can't find the container with id 300f657b0a23c4828d40ac017087f089be7591f69de01ae10b35ad71c174dded
Nov 04 03:16:28.208832 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:28.208809    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f627240_24ef_4aee_9810_51f422083ca3.slice/crio-90b9b99202c8eab287b223a0fbeccb03c667685bb4708c56bc6494a142c655ce WatchSource:0}: Error finding container 90b9b99202c8eab287b223a0fbeccb03c667685bb4708c56bc6494a142c655ce: Status 404 returned error can't find the container with id 90b9b99202c8eab287b223a0fbeccb03c667685bb4708c56bc6494a142c655ce
Nov 04 03:16:28.228033 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:28.228008    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda683bd45_33c3_43fb_9530_292d7fea6970.slice/crio-5424eb6df08c8544680cf425952a24a510af771c90b97047466042e5712bcc27 WatchSource:0}: Error finding container 5424eb6df08c8544680cf425952a24a510af771c90b97047466042e5712bcc27: Status 404 returned error can't find the container with id 5424eb6df08c8544680cf425952a24a510af771c90b97047466042e5712bcc27
Nov 04 03:16:28.250557 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:28.250532    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7713890e_763d_4a51_bffb_6cf8dd12fd18.slice/crio-4354c79b250200492dc51c2675a2335f6c00a5f66b3cd19b4c0c27fe20b3cfd9 WatchSource:0}: Error finding container 4354c79b250200492dc51c2675a2335f6c00a5f66b3cd19b4c0c27fe20b3cfd9: Status 404 returned error can't find the container with id 4354c79b250200492dc51c2675a2335f6c00a5f66b3cd19b4c0c27fe20b3cfd9
Nov 04 03:16:28.499973 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:28.499898    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d10e9268-df07-428b-ae25-528d9fe075d2-original-pull-secret\") pod \"global-pull-secret-syncer-tkkhk\" (UID: \"d10e9268-df07-428b-ae25-528d9fe075d2\") " pod="kube-system/global-pull-secret-syncer-tkkhk"
Nov 04 03:16:28.499973 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:28.499976    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc3d3c0e-2877-4264-93a7-be3756d2dc5c-metrics-certs\") pod \"network-metrics-daemon-lngrr\" (UID: \"fc3d3c0e-2877-4264-93a7-be3756d2dc5c\") " pod="openshift-multus/network-metrics-daemon-lngrr"
Nov 04 03:16:28.500180 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:28.500102    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:16:28.500180 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:28.500162    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc3d3c0e-2877-4264-93a7-be3756d2dc5c-metrics-certs podName:fc3d3c0e-2877-4264-93a7-be3756d2dc5c nodeName:}" failed. No retries permitted until 2026-11-04 03:16:30.500145142 +0000 UTC m=+5.329117666 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fc3d3c0e-2877-4264-93a7-be3756d2dc5c-metrics-certs") pod "network-metrics-daemon-lngrr" (UID: "fc3d3c0e-2877-4264-93a7-be3756d2dc5c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:16:28.500292 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:28.500249    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 03:16:28.500292 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:28.500282    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d10e9268-df07-428b-ae25-528d9fe075d2-original-pull-secret podName:d10e9268-df07-428b-ae25-528d9fe075d2 nodeName:}" failed. No retries permitted until 2026-11-04 03:16:30.500271381 +0000 UTC m=+5.329243906 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d10e9268-df07-428b-ae25-528d9fe075d2-original-pull-secret") pod "global-pull-secret-syncer-tkkhk" (UID: "d10e9268-df07-428b-ae25-528d9fe075d2") : object "kube-system"/"original-pull-secret" not registered
Nov 04 03:16:28.600307 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:28.600268    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jltvv\" (UniqueName: \"kubernetes.io/projected/04b955ef-528b-4573-b18f-0957658f79e4-kube-api-access-jltvv\") pod \"network-check-target-hjssc\" (UID: \"04b955ef-528b-4573-b18f-0957658f79e4\") " pod="openshift-network-diagnostics/network-check-target-hjssc"
Nov 04 03:16:28.600923 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:28.600463    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 03:16:28.600923 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:28.600490    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 03:16:28.600923 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:28.600503    2579 projected.go:196] Error preparing data for projected volume kube-api-access-jltvv for pod openshift-network-diagnostics/network-check-target-hjssc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:16:28.600923 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:28.600568    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/04b955ef-528b-4573-b18f-0957658f79e4-kube-api-access-jltvv podName:04b955ef-528b-4573-b18f-0957658f79e4 nodeName:}" failed. No retries permitted until 2026-11-04 03:16:30.600547425 +0000 UTC m=+5.429519938 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-jltvv" (UniqueName: "kubernetes.io/projected/04b955ef-528b-4573-b18f-0957658f79e4-kube-api-access-jltvv") pod "network-check-target-hjssc" (UID: "04b955ef-528b-4573-b18f-0957658f79e4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:16:28.737046 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:28.737013    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hjssc"
Nov 04 03:16:28.737220 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:28.737135    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hjssc" podUID="04b955ef-528b-4573-b18f-0957658f79e4"
Nov 04 03:16:28.737286 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:28.737251    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkkhk"
Nov 04 03:16:28.737381 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:28.737354    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkkhk" podUID="d10e9268-df07-428b-ae25-528d9fe075d2"
Nov 04 03:16:28.737511 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:28.737451    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lngrr"
Nov 04 03:16:28.737565 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:28.737537    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lngrr" podUID="fc3d3c0e-2877-4264-93a7-be3756d2dc5c"
Nov 04 03:16:28.744742 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:28.744707    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qtt77" event={"ID":"a683bd45-33c3-43fb-9530-292d7fea6970","Type":"ContainerStarted","Data":"5424eb6df08c8544680cf425952a24a510af771c90b97047466042e5712bcc27"}
Nov 04 03:16:28.746202 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:28.746176    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-67bjk" event={"ID":"ff9f5afb-957c-4383-8fa9-698f7ada29b0","Type":"ContainerStarted","Data":"300f657b0a23c4828d40ac017087f089be7591f69de01ae10b35ad71c174dded"}
Nov 04 03:16:28.747596 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:28.747567    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-m2vkr" event={"ID":"029e7b7b-265e-459a-bc09-a90b5c179993","Type":"ContainerStarted","Data":"d732058c584482205f0cda0ced5e810f97ab9a507e40b32658593463a317cfb1"}
Nov 04 03:16:28.750551 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:28.750345    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8" event={"ID":"d604b334-07f4-40f2-bfa9-4b26c301c84c","Type":"ContainerStarted","Data":"460c8b9aec9c460ded874aaca5a63be78eae1fd226e700ffc7f1587fe0828bbd"}
Nov 04 03:16:28.751826 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:28.751800    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-sdhv4" event={"ID":"45621191-5080-4ee9-a08e-f7b2ee49fe41","Type":"ContainerStarted","Data":"f1f81bac5a3c3e1fc69b0c3c653eb547e6c75e948f2c3a1c3b708b5b3139249e"}
Nov 04 03:16:28.753712 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:28.753240    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z" event={"ID":"21a42913-5c4e-4057-aae5-2804038f16e4","Type":"ContainerStarted","Data":"ab7a27f713f1fb60b4c3618a32d929cccdb24f0205e7a1673e82809112865f81"}
Nov 04 03:16:28.754735 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:28.754709    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-n4b5s" event={"ID":"7713890e-763d-4a51-bffb-6cf8dd12fd18","Type":"ContainerStarted","Data":"4354c79b250200492dc51c2675a2335f6c00a5f66b3cd19b4c0c27fe20b3cfd9"}
Nov 04 03:16:28.757111 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:28.757087    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9xgrv" event={"ID":"6f627240-24ef-4aee-9810-51f422083ca3","Type":"ContainerStarted","Data":"90b9b99202c8eab287b223a0fbeccb03c667685bb4708c56bc6494a142c655ce"}
Nov 04 03:16:28.758442 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:28.758412    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6pwhj" event={"ID":"aab0bd9b-cb90-4d9b-9f46-52c6c87a34cf","Type":"ContainerStarted","Data":"f8de65ace441d748bf714ac3874c1d9d5bdca604c8b8c76dba6c256db3d1438d"}
Nov 04 03:16:28.894788 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:28.894722    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 03:11:26 +0000 UTC" deadline="2028-08-06 13:34:31.525967118 +0000 UTC"
Nov 04 03:16:28.894788 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:28.894782    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15394h18m2.631188536s"
Nov 04 03:16:30.521521 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:30.521445    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d10e9268-df07-428b-ae25-528d9fe075d2-original-pull-secret\") pod \"global-pull-secret-syncer-tkkhk\" (UID: \"d10e9268-df07-428b-ae25-528d9fe075d2\") " pod="kube-system/global-pull-secret-syncer-tkkhk"
Nov 04 03:16:30.521521 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:30.521516    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc3d3c0e-2877-4264-93a7-be3756d2dc5c-metrics-certs\") pod \"network-metrics-daemon-lngrr\" (UID: \"fc3d3c0e-2877-4264-93a7-be3756d2dc5c\") " pod="openshift-multus/network-metrics-daemon-lngrr"
Nov 04 03:16:30.522163 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:30.521649    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:16:30.522163 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:30.521718    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc3d3c0e-2877-4264-93a7-be3756d2dc5c-metrics-certs podName:fc3d3c0e-2877-4264-93a7-be3756d2dc5c nodeName:}" failed. No retries permitted until 2026-11-04 03:16:34.521698144 +0000 UTC m=+9.350670653 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fc3d3c0e-2877-4264-93a7-be3756d2dc5c-metrics-certs") pod "network-metrics-daemon-lngrr" (UID: "fc3d3c0e-2877-4264-93a7-be3756d2dc5c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:16:30.522163 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:30.521796    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 03:16:30.522163 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:30.521827    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d10e9268-df07-428b-ae25-528d9fe075d2-original-pull-secret podName:d10e9268-df07-428b-ae25-528d9fe075d2 nodeName:}" failed. No retries permitted until 2026-11-04 03:16:34.521817482 +0000 UTC m=+9.350789994 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d10e9268-df07-428b-ae25-528d9fe075d2-original-pull-secret") pod "global-pull-secret-syncer-tkkhk" (UID: "d10e9268-df07-428b-ae25-528d9fe075d2") : object "kube-system"/"original-pull-secret" not registered
Nov 04 03:16:30.622218 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:30.622177    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jltvv\" (UniqueName: \"kubernetes.io/projected/04b955ef-528b-4573-b18f-0957658f79e4-kube-api-access-jltvv\") pod \"network-check-target-hjssc\" (UID: \"04b955ef-528b-4573-b18f-0957658f79e4\") " pod="openshift-network-diagnostics/network-check-target-hjssc"
Nov 04 03:16:30.622428 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:30.622393    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 03:16:30.622505 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:30.622438    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 03:16:30.622505 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:30.622452    2579 projected.go:196] Error preparing data for projected volume kube-api-access-jltvv for pod openshift-network-diagnostics/network-check-target-hjssc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:16:30.622608 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:30.622509    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/04b955ef-528b-4573-b18f-0957658f79e4-kube-api-access-jltvv podName:04b955ef-528b-4573-b18f-0957658f79e4 nodeName:}" failed. No retries permitted until 2026-11-04 03:16:34.62249169 +0000 UTC m=+9.451464216 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-jltvv" (UniqueName: "kubernetes.io/projected/04b955ef-528b-4573-b18f-0957658f79e4-kube-api-access-jltvv") pod "network-check-target-hjssc" (UID: "04b955ef-528b-4573-b18f-0957658f79e4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:16:30.736936 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:30.736908    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hjssc"
Nov 04 03:16:30.737113 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:30.737034    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hjssc" podUID="04b955ef-528b-4573-b18f-0957658f79e4"
Nov 04 03:16:30.737301 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:30.737283    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkkhk"
Nov 04 03:16:30.737434 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:30.737376    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkkhk" podUID="d10e9268-df07-428b-ae25-528d9fe075d2"
Nov 04 03:16:30.737512 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:30.737443    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lngrr"
Nov 04 03:16:30.737565 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:30.737526    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lngrr" podUID="fc3d3c0e-2877-4264-93a7-be3756d2dc5c"
Nov 04 03:16:30.775039 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:30.774848    2579 generic.go:356] "Generic (PLEG): container finished" podID="2f2b595dbb5c94185c06c3dd3c0c8dce" containerID="24bc55a0ebc85a2d139d06103d000aa6c926738a73bf7f80c7cd444f5253a450" exitCode=0
Nov 04 03:16:30.775039 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:30.774903    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.us-west-2.compute.internal" event={"ID":"2f2b595dbb5c94185c06c3dd3c0c8dce","Type":"ContainerDied","Data":"24bc55a0ebc85a2d139d06103d000aa6c926738a73bf7f80c7cd444f5253a450"}
Nov 04 03:16:31.789824 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:31.789634    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.us-west-2.compute.internal" event={"ID":"2f2b595dbb5c94185c06c3dd3c0c8dce","Type":"ContainerStarted","Data":"cb37a668c8ff0efe8e16b948c8e5073c8e537c8b73c1fdccdbaa3570ac9260fb"}
Nov 04 03:16:31.796691 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:31.796656    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-50.us-west-2.compute.internal" event={"ID":"aeb0f7512125881a2a5e1813eb42a8f7","Type":"ContainerStarted","Data":"9e920e155b78915b58a88d5cd95c09eb1a4ac7057fb6e0b810e2ac157e5defee"}
Nov 04 03:16:32.008571 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:32.008507    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.us-west-2.compute.internal" podStartSLOduration=6.008487146 podStartE2EDuration="6.008487146s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:16:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:16:31.900245563 +0000 UTC m=+6.729218095" watchObservedRunningTime="2026-11-04 03:16:32.008487146 +0000 UTC m=+6.837459679"
Nov 04 03:16:32.008953 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:32.008912    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-50.us-west-2.compute.internal" podStartSLOduration=6.008898902 podStartE2EDuration="6.008898902s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:16:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:16:32.007827365 +0000 UTC m=+6.836799897" watchObservedRunningTime="2026-11-04 03:16:32.008898902 +0000 UTC m=+6.837871437"
Nov 04 03:16:32.737421 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:32.736921    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hjssc"
Nov 04 03:16:32.737421 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:32.737059    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hjssc" podUID="04b955ef-528b-4573-b18f-0957658f79e4"
Nov 04 03:16:32.737421 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:32.737136    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkkhk"
Nov 04 03:16:32.737421 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:32.737211    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkkhk" podUID="d10e9268-df07-428b-ae25-528d9fe075d2"
Nov 04 03:16:32.737421 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:32.737275    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lngrr"
Nov 04 03:16:32.737421 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:32.737358    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lngrr" podUID="fc3d3c0e-2877-4264-93a7-be3756d2dc5c"
Nov 04 03:16:32.807562 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:32.807221    2579 generic.go:356] "Generic (PLEG): container finished" podID="ff9f5afb-957c-4383-8fa9-698f7ada29b0" containerID="e6471b52d954d3714e88a74c51f99cdde0ce2d311927d8583d0f2d12a37f3e9d" exitCode=0
Nov 04 03:16:32.807562 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:32.807316    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-67bjk" event={"ID":"ff9f5afb-957c-4383-8fa9-698f7ada29b0","Type":"ContainerDied","Data":"e6471b52d954d3714e88a74c51f99cdde0ce2d311927d8583d0f2d12a37f3e9d"}
Nov 04 03:16:32.812895 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:32.811678    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-m2vkr" event={"ID":"029e7b7b-265e-459a-bc09-a90b5c179993","Type":"ContainerStarted","Data":"6d976974154fff8b9876ed8bd24f32f3c04004e39cad32a4491a4a83a96dc2d5"}
Nov 04 03:16:33.565773 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:33.565748    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-m2vkr"
Nov 04 03:16:33.816815 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:33.816707    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-n4b5s" event={"ID":"7713890e-763d-4a51-bffb-6cf8dd12fd18","Type":"ContainerStarted","Data":"bcf0a6c3dec3e0c8f397b94ab1a9a6a6beb0df5e825b287cca8bb16d5c16fe77"}
Nov 04 03:16:33.924853 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:33.924792    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-m2vkr" podStartSLOduration=4.590805982 podStartE2EDuration="8.92477387s" totalImagesPullingTime="4.333967888s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:16:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:16:33.075668852 +0000 UTC m=+7.904641384" watchObservedRunningTime="2026-11-04 03:16:33.92477387 +0000 UTC m=+8.753746402"
Nov 04 03:16:33.925617 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:33.925571    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-n4b5s" podStartSLOduration=3.632706785 podStartE2EDuration="8.925557856s" totalImagesPullingTime="5.292851071s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:16:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:16:33.924505568 +0000 UTC m=+8.753478099" watchObservedRunningTime="2026-11-04 03:16:33.925557856 +0000 UTC m=+8.754530392"
Nov 04 03:16:34.565340 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:34.565266    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d10e9268-df07-428b-ae25-528d9fe075d2-original-pull-secret\") pod \"global-pull-secret-syncer-tkkhk\" (UID: \"d10e9268-df07-428b-ae25-528d9fe075d2\") " pod="kube-system/global-pull-secret-syncer-tkkhk"
Nov 04 03:16:34.565340 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:34.565335    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc3d3c0e-2877-4264-93a7-be3756d2dc5c-metrics-certs\") pod \"network-metrics-daemon-lngrr\" (UID: \"fc3d3c0e-2877-4264-93a7-be3756d2dc5c\") " pod="openshift-multus/network-metrics-daemon-lngrr"
Nov 04 03:16:34.565596 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:34.565468    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:16:34.565596 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:34.565524    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc3d3c0e-2877-4264-93a7-be3756d2dc5c-metrics-certs podName:fc3d3c0e-2877-4264-93a7-be3756d2dc5c nodeName:}" failed. No retries permitted until 2026-11-04 03:16:42.565506218 +0000 UTC m=+17.394478744 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fc3d3c0e-2877-4264-93a7-be3756d2dc5c-metrics-certs") pod "network-metrics-daemon-lngrr" (UID: "fc3d3c0e-2877-4264-93a7-be3756d2dc5c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:16:34.565596 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:34.565584    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 03:16:34.565759 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:34.565615    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d10e9268-df07-428b-ae25-528d9fe075d2-original-pull-secret podName:d10e9268-df07-428b-ae25-528d9fe075d2 nodeName:}" failed. No retries permitted until 2026-11-04 03:16:42.565604563 +0000 UTC m=+17.394577087 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d10e9268-df07-428b-ae25-528d9fe075d2-original-pull-secret") pod "global-pull-secret-syncer-tkkhk" (UID: "d10e9268-df07-428b-ae25-528d9fe075d2") : object "kube-system"/"original-pull-secret" not registered
Nov 04 03:16:34.666442 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:34.666395    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jltvv\" (UniqueName: \"kubernetes.io/projected/04b955ef-528b-4573-b18f-0957658f79e4-kube-api-access-jltvv\") pod \"network-check-target-hjssc\" (UID: \"04b955ef-528b-4573-b18f-0957658f79e4\") " pod="openshift-network-diagnostics/network-check-target-hjssc"
Nov 04 03:16:34.666618 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:34.666603    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 03:16:34.666680 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:34.666627    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 03:16:34.666680 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:34.666640    2579 projected.go:196] Error preparing data for projected volume kube-api-access-jltvv for pod openshift-network-diagnostics/network-check-target-hjssc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:16:34.666758 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:34.666692    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/04b955ef-528b-4573-b18f-0957658f79e4-kube-api-access-jltvv podName:04b955ef-528b-4573-b18f-0957658f79e4 nodeName:}" failed. No retries permitted until 2026-11-04 03:16:42.666675928 +0000 UTC m=+17.495648456 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-jltvv" (UniqueName: "kubernetes.io/projected/04b955ef-528b-4573-b18f-0957658f79e4-kube-api-access-jltvv") pod "network-check-target-hjssc" (UID: "04b955ef-528b-4573-b18f-0957658f79e4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:16:34.736963 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:34.736507    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkkhk"
Nov 04 03:16:34.736963 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:34.736640    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkkhk" podUID="d10e9268-df07-428b-ae25-528d9fe075d2"
Nov 04 03:16:34.736963 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:34.736723    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lngrr"
Nov 04 03:16:34.736963 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:34.736810    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lngrr" podUID="fc3d3c0e-2877-4264-93a7-be3756d2dc5c"
Nov 04 03:16:34.736963 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:34.736862    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hjssc"
Nov 04 03:16:34.736963 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:34.736925    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hjssc" podUID="04b955ef-528b-4573-b18f-0957658f79e4"
Nov 04 03:16:34.824698 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:34.824641    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8" event={"ID":"d604b334-07f4-40f2-bfa9-4b26c301c84c","Type":"ContainerStarted","Data":"684ebd1f640883b39e635f4b7f2b8683a812e049b2b0a1bbaa895c98d61cfadd"}
Nov 04 03:16:36.736048 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:36.736017    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkkhk"
Nov 04 03:16:36.736520 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:36.736060    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lngrr"
Nov 04 03:16:36.736520 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:36.736079    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hjssc"
Nov 04 03:16:36.736520 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:36.736179    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lngrr" podUID="fc3d3c0e-2877-4264-93a7-be3756d2dc5c"
Nov 04 03:16:36.736520 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:36.736236    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkkhk" podUID="d10e9268-df07-428b-ae25-528d9fe075d2"
Nov 04 03:16:36.736520 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:36.736274    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hjssc" podUID="04b955ef-528b-4573-b18f-0957658f79e4"
Nov 04 03:16:36.810837 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:36.810811    2579 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 04 03:16:36.832580 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:36.832540    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6pwhj" event={"ID":"aab0bd9b-cb90-4d9b-9f46-52c6c87a34cf","Type":"ContainerStarted","Data":"b2a8aed4cc7b494e55b323095694474b7de9246c30092f4675998b368cf00021"}
Nov 04 03:16:36.835439 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:36.835383    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8" event={"ID":"d604b334-07f4-40f2-bfa9-4b26c301c84c","Type":"ContainerStarted","Data":"d7e33babb74106ec823de20e006362fa6d63838366dfbf029b7a150651607b69"}
Nov 04 03:16:36.837519 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:36.837496    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9xgrv" event={"ID":"6f627240-24ef-4aee-9810-51f422083ca3","Type":"ContainerStarted","Data":"686830cf6c83bdc373a73da6a55e08b5b2eebce9d451a8f88459465e82992392"}
Nov 04 03:16:36.940720 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:36.940654    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-6pwhj" podStartSLOduration=3.748930625 podStartE2EDuration="11.940636231s" totalImagesPullingTime="8.191705606s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:16:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:16:36.94004877 +0000 UTC m=+11.769021302" watchObservedRunningTime="2026-11-04 03:16:36.940636231 +0000 UTC m=+11.769608766"
Nov 04 03:16:37.333669 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:37.333634    2579 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-m2vkr"
Nov 04 03:16:37.334453 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:37.334364    2579 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-m2vkr"
Nov 04 03:16:37.442314 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:37.442252    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-9xgrv" podStartSLOduration=3.202115618 podStartE2EDuration="11.442234051s" totalImagesPullingTime="8.240118433s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:16:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:16:37.062278034 +0000 UTC m=+11.891250567" watchObservedRunningTime="2026-11-04 03:16:37.442234051 +0000 UTC m=+12.271206583"
Nov 04 03:16:37.651535 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:37.651377    2579 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-04T03:16:36.810832647Z","UUID":"d24d74f7-4125-4043-9c9c-60d0fc719694","Handler":null,"Name":"","Endpoint":""}
Nov 04 03:16:37.653307 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:37.653285    2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 04 03:16:37.653435 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:37.653318    2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 04 03:16:37.842534 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:37.842501    2579 generic.go:356] "Generic (PLEG): container finished" podID="ff9f5afb-957c-4383-8fa9-698f7ada29b0" containerID="709c6f80267e0ae83963c9133ee84eb003d734bf58d764285174ddf89e8378de" exitCode=0
Nov 04 03:16:37.842974 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:37.842568    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-67bjk" event={"ID":"ff9f5afb-957c-4383-8fa9-698f7ada29b0","Type":"ContainerDied","Data":"709c6f80267e0ae83963c9133ee84eb003d734bf58d764285174ddf89e8378de"}
Nov 04 03:16:37.844299 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:37.844226    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-m2vkr"
Nov 04 03:16:38.736337 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:38.736128    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hjssc"
Nov 04 03:16:38.736483 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:38.736202    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lngrr"
Nov 04 03:16:38.736483 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:38.736361    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hjssc" podUID="04b955ef-528b-4573-b18f-0957658f79e4"
Nov 04 03:16:38.736483 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:38.736211    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkkhk"
Nov 04 03:16:38.736483 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:38.736461    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lngrr" podUID="fc3d3c0e-2877-4264-93a7-be3756d2dc5c"
Nov 04 03:16:38.736624 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:38.736496    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkkhk" podUID="d10e9268-df07-428b-ae25-528d9fe075d2"
Nov 04 03:16:38.847571 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:38.847536    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8" event={"ID":"d604b334-07f4-40f2-bfa9-4b26c301c84c","Type":"ContainerStarted","Data":"532b3bab0750db09f51b9e5a6259603efffce5edf071b2ee8c720b1fc6db8af1"}
Nov 04 03:16:38.849441 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:38.849396    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-sdhv4" event={"ID":"45621191-5080-4ee9-a08e-f7b2ee49fe41","Type":"ContainerStarted","Data":"837f995ee9dee6f6fbbf42cac9f020a26220cd8063203c0aa32064f66ac9a036"}
Nov 04 03:16:38.850894 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:38.850873    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qtt77" event={"ID":"a683bd45-33c3-43fb-9530-292d7fea6970","Type":"ContainerStarted","Data":"d9a9c7bbb8bd973be247f95875c653412055d152ab11b2037006a4c08edc5aed"}
Nov 04 03:16:38.957530 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:38.957484    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8" podStartSLOduration=3.033033193 podStartE2EDuration="12.957471813s" totalImagesPullingTime="9.92443862s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:16:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:16:38.957257862 +0000 UTC m=+13.786230397" watchObservedRunningTime="2026-11-04 03:16:38.957471813 +0000 UTC m=+13.786444364"
Nov 04 03:16:39.065108 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:39.065024    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-sdhv4" podStartSLOduration=4.655361242 podStartE2EDuration="13.065013123s" totalImagesPullingTime="8.409651881s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:16:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:16:39.064670587 +0000 UTC m=+13.893643116" watchObservedRunningTime="2026-11-04 03:16:39.065013123 +0000 UTC m=+13.893985655"
Nov 04 03:16:39.174840 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:39.174782    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-qtt77" podStartSLOduration=2.916986242 podStartE2EDuration="13.174763763s" totalImagesPullingTime="10.257777521s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:16:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:16:39.174309624 +0000 UTC m=+14.003282158" watchObservedRunningTime="2026-11-04 03:16:39.174763763 +0000 UTC m=+14.003736296"
Nov 04 03:16:39.855055 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:39.854953    2579 generic.go:356] "Generic (PLEG): container finished" podID="ff9f5afb-957c-4383-8fa9-698f7ada29b0" containerID="2d4e3a80926cb5afa449bd6841b511ab58c9bb3dfa1a52e63ea8ed68fbad7e12" exitCode=0
Nov 04 03:16:39.855773 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:39.855081    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-67bjk" event={"ID":"ff9f5afb-957c-4383-8fa9-698f7ada29b0","Type":"ContainerDied","Data":"2d4e3a80926cb5afa449bd6841b511ab58c9bb3dfa1a52e63ea8ed68fbad7e12"}
Nov 04 03:16:40.736197 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:40.736165    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkkhk"
Nov 04 03:16:40.736348 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:40.736236    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hjssc"
Nov 04 03:16:40.736348 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:40.736335    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lngrr"
Nov 04 03:16:40.736450 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:40.736357    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkkhk" podUID="d10e9268-df07-428b-ae25-528d9fe075d2"
Nov 04 03:16:40.736450 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:40.736434    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lngrr" podUID="fc3d3c0e-2877-4264-93a7-be3756d2dc5c"
Nov 04 03:16:40.736550 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:40.736509    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hjssc" podUID="04b955ef-528b-4573-b18f-0957658f79e4"
Nov 04 03:16:41.862245 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:41.862028    2579 generic.go:356] "Generic (PLEG): container finished" podID="ff9f5afb-957c-4383-8fa9-698f7ada29b0" containerID="5e920a96493e2c1f6282e9575be01da804c9971f9cf7815cd96b92e93427e4fd" exitCode=0
Nov 04 03:16:41.862245 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:41.862122    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-67bjk" event={"ID":"ff9f5afb-957c-4383-8fa9-698f7ada29b0","Type":"ContainerDied","Data":"5e920a96493e2c1f6282e9575be01da804c9971f9cf7815cd96b92e93427e4fd"}
Nov 04 03:16:42.639411 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:42.639381    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc3d3c0e-2877-4264-93a7-be3756d2dc5c-metrics-certs\") pod \"network-metrics-daemon-lngrr\" (UID: \"fc3d3c0e-2877-4264-93a7-be3756d2dc5c\") " pod="openshift-multus/network-metrics-daemon-lngrr"
Nov 04 03:16:42.639586 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:42.639459    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d10e9268-df07-428b-ae25-528d9fe075d2-original-pull-secret\") pod \"global-pull-secret-syncer-tkkhk\" (UID: \"d10e9268-df07-428b-ae25-528d9fe075d2\") " pod="kube-system/global-pull-secret-syncer-tkkhk"
Nov 04 03:16:42.639586 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:42.639524    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:16:42.639586 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:42.639570    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 03:16:42.639727 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:42.639598    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc3d3c0e-2877-4264-93a7-be3756d2dc5c-metrics-certs podName:fc3d3c0e-2877-4264-93a7-be3756d2dc5c nodeName:}" failed. No retries permitted until 2026-11-04 03:16:58.639578823 +0000 UTC m=+33.468551342 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fc3d3c0e-2877-4264-93a7-be3756d2dc5c-metrics-certs") pod "network-metrics-daemon-lngrr" (UID: "fc3d3c0e-2877-4264-93a7-be3756d2dc5c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:16:42.639727 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:42.639631    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d10e9268-df07-428b-ae25-528d9fe075d2-original-pull-secret podName:d10e9268-df07-428b-ae25-528d9fe075d2 nodeName:}" failed. No retries permitted until 2026-11-04 03:16:58.639620925 +0000 UTC m=+33.468593468 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d10e9268-df07-428b-ae25-528d9fe075d2-original-pull-secret") pod "global-pull-secret-syncer-tkkhk" (UID: "d10e9268-df07-428b-ae25-528d9fe075d2") : object "kube-system"/"original-pull-secret" not registered
Nov 04 03:16:42.736022 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:42.735986    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hjssc"
Nov 04 03:16:42.736170 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:42.736070    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lngrr"
Nov 04 03:16:42.736238 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:42.736175    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkkhk"
Nov 04 03:16:42.736238 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:42.736183    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hjssc" podUID="04b955ef-528b-4573-b18f-0957658f79e4"
Nov 04 03:16:42.736310 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:42.736289    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkkhk" podUID="d10e9268-df07-428b-ae25-528d9fe075d2"
Nov 04 03:16:42.736427 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:42.736389    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lngrr" podUID="fc3d3c0e-2877-4264-93a7-be3756d2dc5c"
Nov 04 03:16:42.740178 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:42.740150    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jltvv\" (UniqueName: \"kubernetes.io/projected/04b955ef-528b-4573-b18f-0957658f79e4-kube-api-access-jltvv\") pod \"network-check-target-hjssc\" (UID: \"04b955ef-528b-4573-b18f-0957658f79e4\") " pod="openshift-network-diagnostics/network-check-target-hjssc"
Nov 04 03:16:42.740309 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:42.740291    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 03:16:42.740350 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:42.740310    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 03:16:42.740350 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:42.740323    2579 projected.go:196] Error preparing data for projected volume kube-api-access-jltvv for pod openshift-network-diagnostics/network-check-target-hjssc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:16:42.740447 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:42.740366    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/04b955ef-528b-4573-b18f-0957658f79e4-kube-api-access-jltvv podName:04b955ef-528b-4573-b18f-0957658f79e4 nodeName:}" failed. No retries permitted until 2026-11-04 03:16:58.740353845 +0000 UTC m=+33.569326354 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-jltvv" (UniqueName: "kubernetes.io/projected/04b955ef-528b-4573-b18f-0957658f79e4-kube-api-access-jltvv") pod "network-check-target-hjssc" (UID: "04b955ef-528b-4573-b18f-0957658f79e4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:16:44.736918 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:44.736876    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkkhk"
Nov 04 03:16:44.737341 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:44.736876    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hjssc"
Nov 04 03:16:44.737341 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:44.737004    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lngrr"
Nov 04 03:16:44.737341 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:44.737023    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkkhk" podUID="d10e9268-df07-428b-ae25-528d9fe075d2"
Nov 04 03:16:44.737341 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:44.737132    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lngrr" podUID="fc3d3c0e-2877-4264-93a7-be3756d2dc5c"
Nov 04 03:16:44.737341 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:44.737224    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hjssc" podUID="04b955ef-528b-4573-b18f-0957658f79e4"
Nov 04 03:16:44.871522 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:44.871487    2579 generic.go:356] "Generic (PLEG): container finished" podID="ff9f5afb-957c-4383-8fa9-698f7ada29b0" containerID="ea1737077340b39314b28c31e58d589013d2fe974ca30aa9904407df7b66a060" exitCode=0
Nov 04 03:16:44.871692 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:44.871535    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-67bjk" event={"ID":"ff9f5afb-957c-4383-8fa9-698f7ada29b0","Type":"ContainerDied","Data":"ea1737077340b39314b28c31e58d589013d2fe974ca30aa9904407df7b66a060"}
Nov 04 03:16:45.877181 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:45.876943    2579 generic.go:356] "Generic (PLEG): container finished" podID="ff9f5afb-957c-4383-8fa9-698f7ada29b0" containerID="945d7e1f5ca47bb2972f8b9e6ea4ad590723f0f5ebd6d6e7999fffe8d5a9d192" exitCode=0
Nov 04 03:16:45.877642 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:45.877033    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-67bjk" event={"ID":"ff9f5afb-957c-4383-8fa9-698f7ada29b0","Type":"ContainerDied","Data":"945d7e1f5ca47bb2972f8b9e6ea4ad590723f0f5ebd6d6e7999fffe8d5a9d192"}
Nov 04 03:16:45.879966 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:45.879948    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vmw5z_21a42913-5c4e-4057-aae5-2804038f16e4/ovn-acl-logging/0.log"
Nov 04 03:16:45.880285 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:45.880261    2579 generic.go:356] "Generic (PLEG): container finished" podID="21a42913-5c4e-4057-aae5-2804038f16e4" containerID="2f44e914fdba9a07bd980e85975eb79d8885d9caba161a8263e0dce9125701b1" exitCode=1
Nov 04 03:16:45.880384 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:45.880301    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z" event={"ID":"21a42913-5c4e-4057-aae5-2804038f16e4","Type":"ContainerStarted","Data":"82947174e7a679baad0104a9ec591179d933b9a112450192594456fca8367cef"}
Nov 04 03:16:45.880384 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:45.880353    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z" event={"ID":"21a42913-5c4e-4057-aae5-2804038f16e4","Type":"ContainerStarted","Data":"5a0d1b8a0de6db696273c785dec3f5d06313876f40327c1bcc1cd0d73251e253"}
Nov 04 03:16:45.880384 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:45.880383    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z" event={"ID":"21a42913-5c4e-4057-aae5-2804038f16e4","Type":"ContainerStarted","Data":"7646b30538de687761b215e363e5a788291c1c08a1ae9935666f4fc066ed4901"}
Nov 04 03:16:45.880512 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:45.880429    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z" event={"ID":"21a42913-5c4e-4057-aae5-2804038f16e4","Type":"ContainerStarted","Data":"ae02f78b48a7e50d18391ad832e7fed023c59c4d8ea713fc711b40e8d372005f"}
Nov 04 03:16:45.880512 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:45.880456    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z" event={"ID":"21a42913-5c4e-4057-aae5-2804038f16e4","Type":"ContainerDied","Data":"2f44e914fdba9a07bd980e85975eb79d8885d9caba161a8263e0dce9125701b1"}
Nov 04 03:16:45.880512 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:45.880483    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z" event={"ID":"21a42913-5c4e-4057-aae5-2804038f16e4","Type":"ContainerStarted","Data":"2ffc14ac6c452eeccb9317a2ee12d4ed27b33842beaafbca1b73e1852090e8d8"}
Nov 04 03:16:46.736765 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:46.736726    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkkhk"
Nov 04 03:16:46.736765 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:46.736769    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hjssc"
Nov 04 03:16:46.737009 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:46.736844    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkkhk" podUID="d10e9268-df07-428b-ae25-528d9fe075d2"
Nov 04 03:16:46.737009 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:46.736901    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lngrr"
Nov 04 03:16:46.737009 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:46.736991    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lngrr" podUID="fc3d3c0e-2877-4264-93a7-be3756d2dc5c"
Nov 04 03:16:46.737151 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:46.737017    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hjssc" podUID="04b955ef-528b-4573-b18f-0957658f79e4"
Nov 04 03:16:46.885534 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:46.885504    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-67bjk" event={"ID":"ff9f5afb-957c-4383-8fa9-698f7ada29b0","Type":"ContainerStarted","Data":"798a2f1a7bfaa775f9eed822d64a57094b698ffbefff11c1575a875687361e5d"}
Nov 04 03:16:47.000038 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:46.999947    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-67bjk" podStartSLOduration=8.03574594 podStartE2EDuration="20.99993349s" totalImagesPullingTime="12.96418755s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:16:26 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:16:46.999911275 +0000 UTC m=+21.828883806" watchObservedRunningTime="2026-11-04 03:16:46.99993349 +0000 UTC m=+21.828906021"
Nov 04 03:16:47.891381 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:47.891355    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vmw5z_21a42913-5c4e-4057-aae5-2804038f16e4/ovn-acl-logging/0.log"
Nov 04 03:16:47.891813 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:47.891786    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z" event={"ID":"21a42913-5c4e-4057-aae5-2804038f16e4","Type":"ContainerStarted","Data":"4c86597b6748512c897475cda32a42c8fc03fb4ecf0962e3ba4347589bd87972"}
Nov 04 03:16:48.736590 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:48.736557    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkkhk"
Nov 04 03:16:48.736590 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:48.736583    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lngrr"
Nov 04 03:16:48.736797 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:48.736595    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hjssc"
Nov 04 03:16:48.736797 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:48.736669    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkkhk" podUID="d10e9268-df07-428b-ae25-528d9fe075d2"
Nov 04 03:16:48.736879 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:48.736798    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lngrr" podUID="fc3d3c0e-2877-4264-93a7-be3756d2dc5c"
Nov 04 03:16:48.736879 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:48.736865    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hjssc" podUID="04b955ef-528b-4573-b18f-0957658f79e4"
Nov 04 03:16:50.736015 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:50.735982    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkkhk"
Nov 04 03:16:50.736793 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:50.736092    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lngrr"
Nov 04 03:16:50.736793 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:50.736190    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lngrr" podUID="fc3d3c0e-2877-4264-93a7-be3756d2dc5c"
Nov 04 03:16:50.736793 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:50.736086    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hjssc"
Nov 04 03:16:50.736793 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:50.736089    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkkhk" podUID="d10e9268-df07-428b-ae25-528d9fe075d2"
Nov 04 03:16:50.736793 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:50.736276    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hjssc" podUID="04b955ef-528b-4573-b18f-0957658f79e4"
Nov 04 03:16:50.900366 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:50.900338    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vmw5z_21a42913-5c4e-4057-aae5-2804038f16e4/ovn-acl-logging/0.log"
Nov 04 03:16:50.900746 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:50.900713    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z" event={"ID":"21a42913-5c4e-4057-aae5-2804038f16e4","Type":"ContainerStarted","Data":"ef563b11e442f40288dc3410a81a8e6d0a69f33824e1f407dd970c79b1ebc29d"}
Nov 04 03:16:50.900994 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:50.900973    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:50.901102 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:50.901088    2579 scope.go:122] "RemoveContainer" containerID="2f44e914fdba9a07bd980e85975eb79d8885d9caba161a8263e0dce9125701b1"
Nov 04 03:16:50.918176 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:50.918157    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:51.906984 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:51.906956    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vmw5z_21a42913-5c4e-4057-aae5-2804038f16e4/ovn-acl-logging/0.log"
Nov 04 03:16:51.907386 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:51.907291    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z" event={"ID":"21a42913-5c4e-4057-aae5-2804038f16e4","Type":"ContainerStarted","Data":"9d0374c6c9e7f78fc5e57ad4e76994f01ef0e70333311ba685179359aaf948ca"}
Nov 04 03:16:51.907611 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:51.907580    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:51.907740 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:51.907648    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:51.922722 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:51.922698    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:16:52.079777 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:52.079718    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z" podStartSLOduration=8.8925355 podStartE2EDuration="26.079701099s" totalImagesPullingTime="17.187165599s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:16:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:16:52.0789807 +0000 UTC m=+26.907953234" watchObservedRunningTime="2026-11-04 03:16:52.079701099 +0000 UTC m=+26.908673632"
Nov 04 03:16:52.733474 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:52.733444    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tkkhk"]
Nov 04 03:16:52.733657 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:52.733562    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkkhk"
Nov 04 03:16:52.733726 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:52.733667    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkkhk" podUID="d10e9268-df07-428b-ae25-528d9fe075d2"
Nov 04 03:16:52.736058 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:52.736034    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hjssc"]
Nov 04 03:16:52.736308 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:52.736288    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lngrr"
Nov 04 03:16:52.736308 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:52.736300    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hjssc"
Nov 04 03:16:52.736460 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:52.736368    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lngrr" podUID="fc3d3c0e-2877-4264-93a7-be3756d2dc5c"
Nov 04 03:16:52.736460 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:52.736445    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hjssc" podUID="04b955ef-528b-4573-b18f-0957658f79e4"
Nov 04 03:16:52.741107 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:52.741088    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lngrr"]
Nov 04 03:16:52.909890 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:52.909856    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lngrr"
Nov 04 03:16:52.910334 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:52.909983    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lngrr" podUID="fc3d3c0e-2877-4264-93a7-be3756d2dc5c"
Nov 04 03:16:54.736569 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:54.736532    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lngrr"
Nov 04 03:16:54.736569 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:54.736550    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkkhk"
Nov 04 03:16:54.737025 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:54.736542    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hjssc"
Nov 04 03:16:54.737025 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:54.736642    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lngrr" podUID="fc3d3c0e-2877-4264-93a7-be3756d2dc5c"
Nov 04 03:16:54.737025 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:54.736730    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hjssc" podUID="04b955ef-528b-4573-b18f-0957658f79e4"
Nov 04 03:16:54.737025 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:54.736806    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkkhk" podUID="d10e9268-df07-428b-ae25-528d9fe075d2"
Nov 04 03:16:56.736140 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:56.736105    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hjssc"
Nov 04 03:16:56.736556 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:56.736145    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkkhk"
Nov 04 03:16:56.736556 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:56.736214    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hjssc" podUID="04b955ef-528b-4573-b18f-0957658f79e4"
Nov 04 03:16:56.736556 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:56.736255    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkkhk" podUID="d10e9268-df07-428b-ae25-528d9fe075d2"
Nov 04 03:16:56.736556 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:56.736304    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lngrr"
Nov 04 03:16:56.736556 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:56.736428    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lngrr" podUID="fc3d3c0e-2877-4264-93a7-be3756d2dc5c"
Nov 04 03:16:58.515677 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.514492    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-50.us-west-2.compute.internal" event="NodeReady"
Nov 04 03:16:58.515677 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.514646    2579 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Nov 04 03:16:58.623535 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.623449    2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-xpmwt"]
Nov 04 03:16:58.625473 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.625451    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-xpmwt"
Nov 04 03:16:58.651341 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.651310    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc3d3c0e-2877-4264-93a7-be3756d2dc5c-metrics-certs\") pod \"network-metrics-daemon-lngrr\" (UID: \"fc3d3c0e-2877-4264-93a7-be3756d2dc5c\") " pod="openshift-multus/network-metrics-daemon-lngrr"
Nov 04 03:16:58.651509 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.651358    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d10e9268-df07-428b-ae25-528d9fe075d2-original-pull-secret\") pod \"global-pull-secret-syncer-tkkhk\" (UID: \"d10e9268-df07-428b-ae25-528d9fe075d2\") " pod="kube-system/global-pull-secret-syncer-tkkhk"
Nov 04 03:16:58.651509 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:58.651481    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 03:16:58.651509 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:58.651482    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:16:58.651664 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:58.651537    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d10e9268-df07-428b-ae25-528d9fe075d2-original-pull-secret podName:d10e9268-df07-428b-ae25-528d9fe075d2 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:30.651520443 +0000 UTC m=+65.480492955 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d10e9268-df07-428b-ae25-528d9fe075d2-original-pull-secret") pod "global-pull-secret-syncer-tkkhk" (UID: "d10e9268-df07-428b-ae25-528d9fe075d2") : object "kube-system"/"original-pull-secret" not registered
Nov 04 03:16:58.651664 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:58.651551    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc3d3c0e-2877-4264-93a7-be3756d2dc5c-metrics-certs podName:fc3d3c0e-2877-4264-93a7-be3756d2dc5c nodeName:}" failed. No retries permitted until 2026-11-04 03:17:30.651545941 +0000 UTC m=+65.480518450 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fc3d3c0e-2877-4264-93a7-be3756d2dc5c-metrics-certs") pod "network-metrics-daemon-lngrr" (UID: "fc3d3c0e-2877-4264-93a7-be3756d2dc5c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:16:58.660947 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.660917    2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-n6r7s"]
Nov 04 03:16:58.662967 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.662949    2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-mdchx"]
Nov 04 03:16:58.663082 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.663065    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-n6r7s"
Nov 04 03:16:58.664749 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.664724    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-mdchx"
Nov 04 03:16:58.670027 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.670008    2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-js5xd"]
Nov 04 03:16:58.671751 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.671732    2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7cb9f7c89d-wnpll"]
Nov 04 03:16:58.671865 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.671850    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-js5xd"
Nov 04 03:16:58.673727 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.673707    2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dln2k"]
Nov 04 03:16:58.673840 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.673821    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:16:58.676147 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.675722    2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-6c2jp"]
Nov 04 03:16:58.676147 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.675795    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dln2k"
Nov 04 03:16:58.677648 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.677631    2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-gmt9g"]
Nov 04 03:16:58.677799 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.677753    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-6c2jp"
Nov 04 03:16:58.679553 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.679536    2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-79d6dd87ff-n2jnc"]
Nov 04 03:16:58.679665 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.679650    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmt9g"
Nov 04 03:16:58.681368 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.681350    2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-p9989"]
Nov 04 03:16:58.681478 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.681458    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:16:58.683254 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.683235    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-p9989"
Nov 04 03:16:58.708303 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.708275    2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7cfb798487-kbfzk"]
Nov 04 03:16:58.711596 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.711502    2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-4z8fp"]
Nov 04 03:16:58.716043 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.716016    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7cfb798487-kbfzk"
Nov 04 03:16:58.716975 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.716846    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Nov 04 03:16:58.717831 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.717807    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Nov 04 03:16:58.723319 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.722687    2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-tlhng"]
Nov 04 03:16:58.723687 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.723658    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-4z8fp"
Nov 04 03:16:58.726516 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.726497    2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-7c4986d449-hsb49"]
Nov 04 03:16:58.726637 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.726619    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-tlhng"
Nov 04 03:16:58.728316 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.728298    2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-7lkrx"]
Nov 04 03:16:58.728464 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.728447    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-hsb49"
Nov 04 03:16:58.730421 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.730383    2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-s4852"]
Nov 04 03:16:58.730610 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.730586    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-7lkrx"
Nov 04 03:16:58.732447 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.732425    2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-qbskg"]
Nov 04 03:16:58.732550 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.732477    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-s4852"
Nov 04 03:16:58.734339 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.734323    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-xpmwt"]
Nov 04 03:16:58.734438 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.734351    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-mdchx"]
Nov 04 03:16:58.734438 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.734370    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-n6r7s"]
Nov 04 03:16:58.734438 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.734391    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dln2k"]
Nov 04 03:16:58.734438 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.734422    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7cb9f7c89d-wnpll"]
Nov 04 03:16:58.734561 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.734441    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-6c2jp"]
Nov 04 03:16:58.734561 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.734460    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-gmt9g"]
Nov 04 03:16:58.734561 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.734482    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qbskg"
Nov 04 03:16:58.734561 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.734509    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-4z8fp"]
Nov 04 03:16:58.734561 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.734542    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-79d6dd87ff-n2jnc"]
Nov 04 03:16:58.736191 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.736175    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkkhk"
Nov 04 03:16:58.736299 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.736278    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hjssc"
Nov 04 03:16:58.736438 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.736421    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lngrr"
Nov 04 03:16:58.751653 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.751630    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-597ml\" (UniqueName: \"kubernetes.io/projected/10cd06d4-ef1a-4ecf-a72c-e6d0084f3032-kube-api-access-597ml\") pod \"kas-connection-checker-685895c7c9-n6r7s\" (UID: \"10cd06d4-ef1a-4ecf-a72c-e6d0084f3032\") " pod="kube-system/kas-connection-checker-685895c7c9-n6r7s"
Nov 04 03:16:58.751781 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.751674    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jltvv\" (UniqueName: \"kubernetes.io/projected/04b955ef-528b-4573-b18f-0957658f79e4-kube-api-access-jltvv\") pod \"network-check-target-hjssc\" (UID: \"04b955ef-528b-4573-b18f-0957658f79e4\") " pod="openshift-network-diagnostics/network-check-target-hjssc"
Nov 04 03:16:58.751836 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.751776    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66hms\" (UniqueName: \"kubernetes.io/projected/e0c72601-2624-4632-9f15-c19fcc489aae-kube-api-access-66hms\") pod \"kas-connection-checker-685895c7c9-mdchx\" (UID: \"e0c72601-2624-4632-9f15-c19fcc489aae\") " pod="kube-system/kas-connection-checker-685895c7c9-mdchx"
Nov 04 03:16:58.751836 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.751808    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ldp7\" (UniqueName: \"kubernetes.io/projected/a51f26a7-0ec8-4c56-881a-98c6d625478b-kube-api-access-2ldp7\") pod \"kas-connection-checker-685895c7c9-xpmwt\" (UID: \"a51f26a7-0ec8-4c56-881a-98c6d625478b\") " pod="kube-system/kas-connection-checker-685895c7c9-xpmwt"
Nov 04 03:16:58.753522 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.753482    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Nov 04 03:16:58.753721 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.753610    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-2z9f6\""
Nov 04 03:16:58.753721 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.753642    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Nov 04 03:16:58.753721 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.753671    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Nov 04 03:16:58.754109 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.754091    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-gl4fp\""
Nov 04 03:16:58.754246 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.754194    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Nov 04 03:16:58.756043 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.756021    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7cfb798487-kbfzk"]
Nov 04 03:16:58.756132 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.756074    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-js5xd"]
Nov 04 03:16:58.756844 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.756824    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-7lkrx"]
Nov 04 03:16:58.756886 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.756870    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qbskg"]
Nov 04 03:16:58.764237 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.764210    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-hsb49"]
Nov 04 03:16:58.764340 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.764267    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-s4852"]
Nov 04 03:16:58.765673 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.765648    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-p9989"]
Nov 04 03:16:58.774673 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.774653    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Nov 04 03:16:58.774806 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.774665    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Nov 04 03:16:58.774806 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.774688    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Nov 04 03:16:58.774917 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.774767    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Nov 04 03:16:58.774917 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.774857    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Nov 04 03:16:58.774917 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.774892    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Nov 04 03:16:58.775035 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.775002    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Nov 04 03:16:58.775035 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.775006    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 04 03:16:58.775035 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.775019    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Nov 04 03:16:58.775139 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.775063    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 04 03:16:58.775189 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.775152    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Nov 04 03:16:58.775189 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.775157    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Nov 04 03:16:58.775277 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.775188    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Nov 04 03:16:58.775277 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.775217    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Nov 04 03:16:58.775277 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.775259    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Nov 04 03:16:58.775426 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.775317    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Nov 04 03:16:58.776337 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.776310    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-tlhng"]
Nov 04 03:16:58.821371 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.821350    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Nov 04 03:16:58.822537 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.822502    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Nov 04 03:16:58.822633 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.822551    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 04 03:16:58.822633 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.822565    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 04 03:16:58.822760 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.822647    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Nov 04 03:16:58.822760 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.822698    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Nov 04 03:16:58.822760 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.822741    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vjndb\""
Nov 04 03:16:58.822906 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.822764    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Nov 04 03:16:58.822906 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.822785    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Nov 04 03:16:58.822906 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.822819    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Nov 04 03:16:58.822906 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.822824    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Nov 04 03:16:58.822906 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.822831    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-f4sxh\""
Nov 04 03:16:58.822906 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.822857    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Nov 04 03:16:58.822906 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.822872    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Nov 04 03:16:58.822906 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.822897    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-mnqmt\""
Nov 04 03:16:58.822906 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.822909    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-zsv6g\""
Nov 04 03:16:58.823193 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.822924    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Nov 04 03:16:58.823193 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.823047    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 04 03:16:58.823193 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.823097    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Nov 04 03:16:58.823193 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.823147    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-2nc4r\""
Nov 04 03:16:58.823193 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.823169    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 04 03:16:58.823193 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.823194    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xwzp4\""
Nov 04 03:16:58.823355 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.823186    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-mhdb8\""
Nov 04 03:16:58.852574 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.852544    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fc612506-b699-44bc-a81b-06fee4e819fe-image-registry-private-configuration\") pod \"image-registry-7cb9f7c89d-wnpll\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") " pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:16:58.852740 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.852582    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f1f61b42-3b07-4892-90a7-2efa024e435b-trusted-ca\") pod \"image-registry-79d6dd87ff-n2jnc\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") " pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:16:58.852740 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.852611    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfc98\" (UniqueName: \"kubernetes.io/projected/fc612506-b699-44bc-a81b-06fee4e819fe-kube-api-access-bfc98\") pod \"image-registry-7cb9f7c89d-wnpll\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") " pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:16:58.852740 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.852666    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f1f61b42-3b07-4892-90a7-2efa024e435b-registry-certificates\") pod \"image-registry-79d6dd87ff-n2jnc\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") " pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:16:58.852740 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.852706    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b178ce9-e140-4420-81f1-627de00b90c6-service-ca-bundle\") pod \"insights-operator-7c4986d449-hsb49\" (UID: \"3b178ce9-e140-4420-81f1-627de00b90c6\") " pod="openshift-insights/insights-operator-7c4986d449-hsb49"
Nov 04 03:16:58.852920 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.852740    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/068619f8-7fbb-4b22-ac7d-a1f144e084c0-serving-cert\") pod \"console-operator-7cd56b8dbc-p9989\" (UID: \"068619f8-7fbb-4b22-ac7d-a1f144e084c0\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-p9989"
Nov 04 03:16:58.852920 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.852764    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/06698c23-887e-4ee5-9728-04098200c13d-metrics-tls\") pod \"dns-default-s4852\" (UID: \"06698c23-887e-4ee5-9728-04098200c13d\") " pod="openshift-dns/dns-default-s4852"
Nov 04 03:16:58.852920 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.852789    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3b178ce9-e140-4420-81f1-627de00b90c6-serving-cert\") pod \"insights-operator-7c4986d449-hsb49\" (UID: \"3b178ce9-e140-4420-81f1-627de00b90c6\") " pod="openshift-insights/insights-operator-7c4986d449-hsb49"
Nov 04 03:16:58.852920 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.852834    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e487dae4-8ac9-4acd-a8ad-a0d149909475-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-4z8fp\" (UID: \"e487dae4-8ac9-4acd-a8ad-a0d149909475\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-4z8fp"
Nov 04 03:16:58.852920 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.852891    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f1f61b42-3b07-4892-90a7-2efa024e435b-installation-pull-secrets\") pod \"image-registry-79d6dd87ff-n2jnc\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") " pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:16:58.853124 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.852930    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d731646d-23ab-42d7-ae48-974929b3d2dc-nginx-conf\") pod \"networking-console-plugin-86c478bb88-tlhng\" (UID: \"d731646d-23ab-42d7-ae48-974929b3d2dc\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-tlhng"
Nov 04 03:16:58.853124 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.852975    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fc612506-b699-44bc-a81b-06fee4e819fe-ca-trust-extracted\") pod \"image-registry-7cb9f7c89d-wnpll\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") " pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:16:58.853124 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.852998    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlxxz\" (UniqueName: \"kubernetes.io/projected/e487dae4-8ac9-4acd-a8ad-a0d149909475-kube-api-access-vlxxz\") pod \"cluster-samples-operator-6ddb6c8b94-4z8fp\" (UID: \"e487dae4-8ac9-4acd-a8ad-a0d149909475\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-4z8fp"
Nov 04 03:16:58.853124 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.853026    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f1f61b42-3b07-4892-90a7-2efa024e435b-registry-tls\") pod \"image-registry-79d6dd87ff-n2jnc\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") " pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:16:58.853124 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.853087    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66hms\" (UniqueName: \"kubernetes.io/projected/e0c72601-2624-4632-9f15-c19fcc489aae-kube-api-access-66hms\") pod \"kas-connection-checker-685895c7c9-mdchx\" (UID: \"e0c72601-2624-4632-9f15-c19fcc489aae\") " pod="kube-system/kas-connection-checker-685895c7c9-mdchx"
Nov 04 03:16:58.853124 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.853103    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f1f61b42-3b07-4892-90a7-2efa024e435b-bound-sa-token\") pod \"image-registry-79d6dd87ff-n2jnc\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") " pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:16:58.853124 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.853118    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6dedccdc-aea2-4664-8480-ca3c9359437d-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-dln2k\" (UID: \"6dedccdc-aea2-4664-8480-ca3c9359437d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dln2k"
Nov 04 03:16:58.853348 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.853138    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0beb091e-5c2f-4136-93a2-771f9b861411-serving-cert\") pod \"service-ca-operator-544d749c8c-js5xd\" (UID: \"0beb091e-5c2f-4136-93a2-771f9b861411\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-js5xd"
Nov 04 03:16:58.853348 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.853154    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsjms\" (UniqueName: \"kubernetes.io/projected/c1b21c41-4b51-4da8-8920-c38cf49554bf-kube-api-access-fsjms\") pod \"router-default-7cfb798487-kbfzk\" (UID: \"c1b21c41-4b51-4da8-8920-c38cf49554bf\") " pod="openshift-ingress/router-default-7cfb798487-kbfzk"
Nov 04 03:16:58.853348 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.853198    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8g8qf\" (UniqueName: \"kubernetes.io/projected/e451b329-bd05-44f5-9544-8283f76df9d6-kube-api-access-8g8qf\") pod \"volume-data-source-validator-58c9f5f46d-6c2jp\" (UID: \"e451b329-bd05-44f5-9544-8283f76df9d6\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-6c2jp"
Nov 04 03:16:58.853348 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.853243    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbrl5\" (UniqueName: \"kubernetes.io/projected/e87dded5-1e1b-45cb-ab2e-ae12550dfca7-kube-api-access-sbrl5\") pod \"cluster-monitoring-operator-884576c57-gmt9g\" (UID: \"e87dded5-1e1b-45cb-ab2e-ae12550dfca7\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmt9g"
Nov 04 03:16:58.853348 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.853310    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-597ml\" (UniqueName: \"kubernetes.io/projected/10cd06d4-ef1a-4ecf-a72c-e6d0084f3032-kube-api-access-597ml\") pod \"kas-connection-checker-685895c7c9-n6r7s\" (UID: \"10cd06d4-ef1a-4ecf-a72c-e6d0084f3032\") " pod="kube-system/kas-connection-checker-685895c7c9-n6r7s"
Nov 04 03:16:58.853348 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.853340    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fc612506-b699-44bc-a81b-06fee4e819fe-trusted-ca\") pod \"image-registry-7cb9f7c89d-wnpll\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") " pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:16:58.853618 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.853360    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f1f61b42-3b07-4892-90a7-2efa024e435b-image-registry-private-configuration\") pod \"image-registry-79d6dd87ff-n2jnc\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") " pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:16:58.853618 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.853440    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/06698c23-887e-4ee5-9728-04098200c13d-config-volume\") pod \"dns-default-s4852\" (UID: \"06698c23-887e-4ee5-9728-04098200c13d\") " pod="openshift-dns/dns-default-s4852"
Nov 04 03:16:58.853618 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.853472    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e87dded5-1e1b-45cb-ab2e-ae12550dfca7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-gmt9g\" (UID: \"e87dded5-1e1b-45cb-ab2e-ae12550dfca7\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmt9g"
Nov 04 03:16:58.853618 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.853498    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4pj6\" (UniqueName: \"kubernetes.io/projected/0beb091e-5c2f-4136-93a2-771f9b861411-kube-api-access-m4pj6\") pod \"service-ca-operator-544d749c8c-js5xd\" (UID: \"0beb091e-5c2f-4136-93a2-771f9b861411\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-js5xd"
Nov 04 03:16:58.853618 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.853528    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3b178ce9-e140-4420-81f1-627de00b90c6-tmp\") pod \"insights-operator-7c4986d449-hsb49\" (UID: \"3b178ce9-e140-4420-81f1-627de00b90c6\") " pod="openshift-insights/insights-operator-7c4986d449-hsb49"
Nov 04 03:16:58.853618 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.853589    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1b21c41-4b51-4da8-8920-c38cf49554bf-service-ca-bundle\") pod \"router-default-7cfb798487-kbfzk\" (UID: \"c1b21c41-4b51-4da8-8920-c38cf49554bf\") " pod="openshift-ingress/router-default-7cfb798487-kbfzk"
Nov 04 03:16:58.853791 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.853639    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/068619f8-7fbb-4b22-ac7d-a1f144e084c0-config\") pod \"console-operator-7cd56b8dbc-p9989\" (UID: \"068619f8-7fbb-4b22-ac7d-a1f144e084c0\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-p9989"
Nov 04 03:16:58.853791 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.853664    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06698c23-887e-4ee5-9728-04098200c13d-tmp-dir\") pod \"dns-default-s4852\" (UID: \"06698c23-887e-4ee5-9728-04098200c13d\") " pod="openshift-dns/dns-default-s4852"
Nov 04 03:16:58.853791 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.853682    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/068619f8-7fbb-4b22-ac7d-a1f144e084c0-trusted-ca\") pod \"console-operator-7cd56b8dbc-p9989\" (UID: \"068619f8-7fbb-4b22-ac7d-a1f144e084c0\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-p9989"
Nov 04 03:16:58.853791 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.853704    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fc612506-b699-44bc-a81b-06fee4e819fe-registry-tls\") pod \"image-registry-7cb9f7c89d-wnpll\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") " pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:16:58.853791 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.853738    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e87dded5-1e1b-45cb-ab2e-ae12550dfca7-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-gmt9g\" (UID: \"e87dded5-1e1b-45cb-ab2e-ae12550dfca7\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmt9g"
Nov 04 03:16:58.853791 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.853771    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5h469\" (UniqueName: \"kubernetes.io/projected/55c4af4c-188b-495b-9052-5700a1f60602-kube-api-access-5h469\") pod \"ingress-canary-qbskg\" (UID: \"55c4af4c-188b-495b-9052-5700a1f60602\") " pod="openshift-ingress-canary/ingress-canary-qbskg"
Nov 04 03:16:58.853791 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.853789    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0beb091e-5c2f-4136-93a2-771f9b861411-config\") pod \"service-ca-operator-544d749c8c-js5xd\" (UID: \"0beb091e-5c2f-4136-93a2-771f9b861411\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-js5xd"
Nov 04 03:16:58.854024 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.853853    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fc612506-b699-44bc-a81b-06fee4e819fe-registry-certificates\") pod \"image-registry-7cb9f7c89d-wnpll\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") " pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:16:58.854024 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.853874    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d731646d-23ab-42d7-ae48-974929b3d2dc-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-tlhng\" (UID: \"d731646d-23ab-42d7-ae48-974929b3d2dc\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-tlhng"
Nov 04 03:16:58.854024 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.853893    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fc612506-b699-44bc-a81b-06fee4e819fe-bound-sa-token\") pod \"image-registry-7cb9f7c89d-wnpll\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") " pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:16:58.854024 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.853928    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/e487dae4-8ac9-4acd-a8ad-a0d149909475-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-4z8fp\" (UID: \"e487dae4-8ac9-4acd-a8ad-a0d149909475\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-4z8fp"
Nov 04 03:16:58.854024 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.853950    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/55c4af4c-188b-495b-9052-5700a1f60602-cert\") pod \"ingress-canary-qbskg\" (UID: \"55c4af4c-188b-495b-9052-5700a1f60602\") " pod="openshift-ingress-canary/ingress-canary-qbskg"
Nov 04 03:16:58.854024 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.853999    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fc612506-b699-44bc-a81b-06fee4e819fe-installation-pull-secrets\") pod \"image-registry-7cb9f7c89d-wnpll\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") " pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:16:58.854225 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.854039    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ldp7\" (UniqueName: \"kubernetes.io/projected/a51f26a7-0ec8-4c56-881a-98c6d625478b-kube-api-access-2ldp7\") pod \"kas-connection-checker-685895c7c9-xpmwt\" (UID: \"a51f26a7-0ec8-4c56-881a-98c6d625478b\") " pod="kube-system/kas-connection-checker-685895c7c9-xpmwt"
Nov 04 03:16:58.854225 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.854081    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1b21c41-4b51-4da8-8920-c38cf49554bf-metrics-certs\") pod \"router-default-7cfb798487-kbfzk\" (UID: \"c1b21c41-4b51-4da8-8920-c38cf49554bf\") " pod="openshift-ingress/router-default-7cfb798487-kbfzk"
Nov 04 03:16:58.854225 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.854118    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnrbg\" (UniqueName: \"kubernetes.io/projected/cc469246-2293-4301-9e46-bf9d609be964-kube-api-access-cnrbg\") pod \"network-check-source-6f78d59f99-7lkrx\" (UID: \"cc469246-2293-4301-9e46-bf9d609be964\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-7lkrx"
Nov 04 03:16:58.854225 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.854196    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f1f61b42-3b07-4892-90a7-2efa024e435b-ca-trust-extracted\") pod \"image-registry-79d6dd87ff-n2jnc\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") " pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:16:58.854356 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.854223    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c1b21c41-4b51-4da8-8920-c38cf49554bf-default-certificate\") pod \"router-default-7cfb798487-kbfzk\" (UID: \"c1b21c41-4b51-4da8-8920-c38cf49554bf\") " pod="openshift-ingress/router-default-7cfb798487-kbfzk"
Nov 04 03:16:58.854356 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.854258    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjtnj\" (UniqueName: \"kubernetes.io/projected/f1f61b42-3b07-4892-90a7-2efa024e435b-kube-api-access-kjtnj\") pod \"image-registry-79d6dd87ff-n2jnc\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") " pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:16:58.854356 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.854283    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b178ce9-e140-4420-81f1-627de00b90c6-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-hsb49\" (UID: \"3b178ce9-e140-4420-81f1-627de00b90c6\") " pod="openshift-insights/insights-operator-7c4986d449-hsb49"
Nov 04 03:16:58.854356 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.854310    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcm4d\" (UniqueName: \"kubernetes.io/projected/068619f8-7fbb-4b22-ac7d-a1f144e084c0-kube-api-access-mcm4d\") pod \"console-operator-7cd56b8dbc-p9989\" (UID: \"068619f8-7fbb-4b22-ac7d-a1f144e084c0\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-p9989"
Nov 04 03:16:58.854356 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.854335    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mbsz\" (UniqueName: \"kubernetes.io/projected/6dedccdc-aea2-4664-8480-ca3c9359437d-kube-api-access-5mbsz\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-dln2k\" (UID: \"6dedccdc-aea2-4664-8480-ca3c9359437d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dln2k"
Nov 04 03:16:58.854587 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.854368    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4sqp\" (UniqueName: \"kubernetes.io/projected/06698c23-887e-4ee5-9728-04098200c13d-kube-api-access-v4sqp\") pod \"dns-default-s4852\" (UID: \"06698c23-887e-4ee5-9728-04098200c13d\") " pod="openshift-dns/dns-default-s4852"
Nov 04 03:16:58.854587 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.854446    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6dedccdc-aea2-4664-8480-ca3c9359437d-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-dln2k\" (UID: \"6dedccdc-aea2-4664-8480-ca3c9359437d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dln2k"
Nov 04 03:16:58.854587 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.854503    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c1b21c41-4b51-4da8-8920-c38cf49554bf-stats-auth\") pod \"router-default-7cfb798487-kbfzk\" (UID: \"c1b21c41-4b51-4da8-8920-c38cf49554bf\") " pod="openshift-ingress/router-default-7cfb798487-kbfzk"
Nov 04 03:16:58.854587 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.854545    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghplp\" (UniqueName: \"kubernetes.io/projected/3b178ce9-e140-4420-81f1-627de00b90c6-kube-api-access-ghplp\") pod \"insights-operator-7c4986d449-hsb49\" (UID: \"3b178ce9-e140-4420-81f1-627de00b90c6\") " pod="openshift-insights/insights-operator-7c4986d449-hsb49"
Nov 04 03:16:58.854587 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.854576    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3b178ce9-e140-4420-81f1-627de00b90c6-snapshots\") pod \"insights-operator-7c4986d449-hsb49\" (UID: \"3b178ce9-e140-4420-81f1-627de00b90c6\") " pod="openshift-insights/insights-operator-7c4986d449-hsb49"
Nov 04 03:16:58.875297 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.875219    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Nov 04 03:16:58.875297 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.875269    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 04 03:16:58.875297 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.875270    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 04 03:16:58.875510 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.875294    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Nov 04 03:16:58.875510 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.875300    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 04 03:16:58.875510 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.875392    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Nov 04 03:16:58.875510 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.875428    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 04 03:16:58.875510 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.875437    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Nov 04 03:16:58.875510 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.875470    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 04 03:16:58.875510 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.875470    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Nov 04 03:16:58.875510 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.875487    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 04 03:16:58.875510 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.875505    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 04 03:16:58.876838 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.876820    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Nov 04 03:16:58.886065 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.886043    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jltvv\" (UniqueName: \"kubernetes.io/projected/04b955ef-528b-4573-b18f-0957658f79e4-kube-api-access-jltvv\") pod \"network-check-target-hjssc\" (UID: \"04b955ef-528b-4573-b18f-0957658f79e4\") " pod="openshift-network-diagnostics/network-check-target-hjssc"
Nov 04 03:16:58.911269 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.911236    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-66hms\" (UniqueName: \"kubernetes.io/projected/e0c72601-2624-4632-9f15-c19fcc489aae-kube-api-access-66hms\") pod \"kas-connection-checker-685895c7c9-mdchx\" (UID: \"e0c72601-2624-4632-9f15-c19fcc489aae\") " pod="kube-system/kas-connection-checker-685895c7c9-mdchx"
Nov 04 03:16:58.911269 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.911247    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ldp7\" (UniqueName: \"kubernetes.io/projected/a51f26a7-0ec8-4c56-881a-98c6d625478b-kube-api-access-2ldp7\") pod \"kas-connection-checker-685895c7c9-xpmwt\" (UID: \"a51f26a7-0ec8-4c56-881a-98c6d625478b\") " pod="kube-system/kas-connection-checker-685895c7c9-xpmwt"
Nov 04 03:16:58.911442 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.911293    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-597ml\" (UniqueName: \"kubernetes.io/projected/10cd06d4-ef1a-4ecf-a72c-e6d0084f3032-kube-api-access-597ml\") pod \"kas-connection-checker-685895c7c9-n6r7s\" (UID: \"10cd06d4-ef1a-4ecf-a72c-e6d0084f3032\") " pod="kube-system/kas-connection-checker-685895c7c9-n6r7s"
Nov 04 03:16:58.936265 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.936235    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-xpmwt"
Nov 04 03:16:58.955600 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.955563    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsjms\" (UniqueName: \"kubernetes.io/projected/c1b21c41-4b51-4da8-8920-c38cf49554bf-kube-api-access-fsjms\") pod \"router-default-7cfb798487-kbfzk\" (UID: \"c1b21c41-4b51-4da8-8920-c38cf49554bf\") " pod="openshift-ingress/router-default-7cfb798487-kbfzk"
Nov 04 03:16:58.955722 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.955606    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8g8qf\" (UniqueName: \"kubernetes.io/projected/e451b329-bd05-44f5-9544-8283f76df9d6-kube-api-access-8g8qf\") pod \"volume-data-source-validator-58c9f5f46d-6c2jp\" (UID: \"e451b329-bd05-44f5-9544-8283f76df9d6\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-6c2jp"
Nov 04 03:16:58.955722 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.955634    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbrl5\" (UniqueName: \"kubernetes.io/projected/e87dded5-1e1b-45cb-ab2e-ae12550dfca7-kube-api-access-sbrl5\") pod \"cluster-monitoring-operator-884576c57-gmt9g\" (UID: \"e87dded5-1e1b-45cb-ab2e-ae12550dfca7\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmt9g"
Nov 04 03:16:58.955722 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.955663    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fc612506-b699-44bc-a81b-06fee4e819fe-trusted-ca\") pod \"image-registry-7cb9f7c89d-wnpll\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") " pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:16:58.955722 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.955688    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f1f61b42-3b07-4892-90a7-2efa024e435b-image-registry-private-configuration\") pod \"image-registry-79d6dd87ff-n2jnc\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") " pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:16:58.955936 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.955741    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/06698c23-887e-4ee5-9728-04098200c13d-config-volume\") pod \"dns-default-s4852\" (UID: \"06698c23-887e-4ee5-9728-04098200c13d\") " pod="openshift-dns/dns-default-s4852"
Nov 04 03:16:58.955936 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.955765    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e87dded5-1e1b-45cb-ab2e-ae12550dfca7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-gmt9g\" (UID: \"e87dded5-1e1b-45cb-ab2e-ae12550dfca7\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmt9g"
Nov 04 03:16:58.955936 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.955791    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4pj6\" (UniqueName: \"kubernetes.io/projected/0beb091e-5c2f-4136-93a2-771f9b861411-kube-api-access-m4pj6\") pod \"service-ca-operator-544d749c8c-js5xd\" (UID: \"0beb091e-5c2f-4136-93a2-771f9b861411\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-js5xd"
Nov 04 03:16:58.955936 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.955815    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3b178ce9-e140-4420-81f1-627de00b90c6-tmp\") pod \"insights-operator-7c4986d449-hsb49\" (UID: \"3b178ce9-e140-4420-81f1-627de00b90c6\") " pod="openshift-insights/insights-operator-7c4986d449-hsb49"
Nov 04 03:16:58.955936 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.955853    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1b21c41-4b51-4da8-8920-c38cf49554bf-service-ca-bundle\") pod \"router-default-7cfb798487-kbfzk\" (UID: \"c1b21c41-4b51-4da8-8920-c38cf49554bf\") " pod="openshift-ingress/router-default-7cfb798487-kbfzk"
Nov 04 03:16:58.955936 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.955881    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/068619f8-7fbb-4b22-ac7d-a1f144e084c0-config\") pod \"console-operator-7cd56b8dbc-p9989\" (UID: \"068619f8-7fbb-4b22-ac7d-a1f144e084c0\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-p9989"
Nov 04 03:16:58.955936 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.955900    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06698c23-887e-4ee5-9728-04098200c13d-tmp-dir\") pod \"dns-default-s4852\" (UID: \"06698c23-887e-4ee5-9728-04098200c13d\") " pod="openshift-dns/dns-default-s4852"
Nov 04 03:16:58.955936 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.955925    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/068619f8-7fbb-4b22-ac7d-a1f144e084c0-trusted-ca\") pod \"console-operator-7cd56b8dbc-p9989\" (UID: \"068619f8-7fbb-4b22-ac7d-a1f144e084c0\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-p9989"
Nov 04 03:16:58.955936 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:58.955928    2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 04 03:16:58.956343 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.955948    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fc612506-b699-44bc-a81b-06fee4e819fe-registry-tls\") pod \"image-registry-7cb9f7c89d-wnpll\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") " pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:16:58.956343 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.955975    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e87dded5-1e1b-45cb-ab2e-ae12550dfca7-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-gmt9g\" (UID: \"e87dded5-1e1b-45cb-ab2e-ae12550dfca7\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmt9g"
Nov 04 03:16:58.956343 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:58.955996    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e87dded5-1e1b-45cb-ab2e-ae12550dfca7-cluster-monitoring-operator-tls podName:e87dded5-1e1b-45cb-ab2e-ae12550dfca7 nodeName:}" failed. No retries permitted until 2026-11-04 03:16:59.455974886 +0000 UTC m=+34.284947408 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e87dded5-1e1b-45cb-ab2e-ae12550dfca7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-gmt9g" (UID: "e87dded5-1e1b-45cb-ab2e-ae12550dfca7") : secret "cluster-monitoring-operator-tls" not found
Nov 04 03:16:58.956343 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.956033    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5h469\" (UniqueName: \"kubernetes.io/projected/55c4af4c-188b-495b-9052-5700a1f60602-kube-api-access-5h469\") pod \"ingress-canary-qbskg\" (UID: \"55c4af4c-188b-495b-9052-5700a1f60602\") " pod="openshift-ingress-canary/ingress-canary-qbskg"
Nov 04 03:16:58.956343 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.956064    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0beb091e-5c2f-4136-93a2-771f9b861411-config\") pod \"service-ca-operator-544d749c8c-js5xd\" (UID: \"0beb091e-5c2f-4136-93a2-771f9b861411\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-js5xd"
Nov 04 03:16:58.956343 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.956112    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fc612506-b699-44bc-a81b-06fee4e819fe-registry-certificates\") pod \"image-registry-7cb9f7c89d-wnpll\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") " pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:16:58.956676 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.956637    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/06698c23-887e-4ee5-9728-04098200c13d-config-volume\") pod \"dns-default-s4852\" (UID: \"06698c23-887e-4ee5-9728-04098200c13d\") " pod="openshift-dns/dns-default-s4852"
Nov 04 03:16:58.956760 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.956724    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0beb091e-5c2f-4136-93a2-771f9b861411-config\") pod \"service-ca-operator-544d749c8c-js5xd\" (UID: \"0beb091e-5c2f-4136-93a2-771f9b861411\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-js5xd"
Nov 04 03:16:58.956835 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.956809    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e87dded5-1e1b-45cb-ab2e-ae12550dfca7-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-gmt9g\" (UID: \"e87dded5-1e1b-45cb-ab2e-ae12550dfca7\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmt9g"
Nov 04 03:16:58.956981 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.956871    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d731646d-23ab-42d7-ae48-974929b3d2dc-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-tlhng\" (UID: \"d731646d-23ab-42d7-ae48-974929b3d2dc\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-tlhng"
Nov 04 03:16:58.956981 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.956914    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fc612506-b699-44bc-a81b-06fee4e819fe-bound-sa-token\") pod \"image-registry-7cb9f7c89d-wnpll\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") " pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:16:58.956981 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:58.956931    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 03:16:58.956981 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.956940    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/e487dae4-8ac9-4acd-a8ad-a0d149909475-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-4z8fp\" (UID: \"e487dae4-8ac9-4acd-a8ad-a0d149909475\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-4z8fp"
Nov 04 03:16:58.956981 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:58.956948    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cb9f7c89d-wnpll: secret "image-registry-tls" not found
Nov 04 03:16:58.956981 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.956966    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/55c4af4c-188b-495b-9052-5700a1f60602-cert\") pod \"ingress-canary-qbskg\" (UID: \"55c4af4c-188b-495b-9052-5700a1f60602\") " pod="openshift-ingress-canary/ingress-canary-qbskg"
Nov 04 03:16:58.957530 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:58.956990    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c1b21c41-4b51-4da8-8920-c38cf49554bf-service-ca-bundle podName:c1b21c41-4b51-4da8-8920-c38cf49554bf nodeName:}" failed. No retries permitted until 2026-11-04 03:16:59.456972745 +0000 UTC m=+34.285945266 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c1b21c41-4b51-4da8-8920-c38cf49554bf-service-ca-bundle") pod "router-default-7cfb798487-kbfzk" (UID: "c1b21c41-4b51-4da8-8920-c38cf49554bf") : configmap references non-existent config key: service-ca.crt
Nov 04 03:16:58.957530 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.957005    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3b178ce9-e140-4420-81f1-627de00b90c6-tmp\") pod \"insights-operator-7c4986d449-hsb49\" (UID: \"3b178ce9-e140-4420-81f1-627de00b90c6\") " pod="openshift-insights/insights-operator-7c4986d449-hsb49"
Nov 04 03:16:58.957530 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.957104    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06698c23-887e-4ee5-9728-04098200c13d-tmp-dir\") pod \"dns-default-s4852\" (UID: \"06698c23-887e-4ee5-9728-04098200c13d\") " pod="openshift-dns/dns-default-s4852"
Nov 04 03:16:58.957530 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:58.957130    2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 04 03:16:58.957530 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:58.957163    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fc612506-b699-44bc-a81b-06fee4e819fe-registry-tls podName:fc612506-b699-44bc-a81b-06fee4e819fe nodeName:}" failed. No retries permitted until 2026-11-04 03:16:59.457149433 +0000 UTC m=+34.286121944 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fc612506-b699-44bc-a81b-06fee4e819fe-registry-tls") pod "image-registry-7cb9f7c89d-wnpll" (UID: "fc612506-b699-44bc-a81b-06fee4e819fe") : secret "image-registry-tls" not found
Nov 04 03:16:58.957530 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.957133    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/068619f8-7fbb-4b22-ac7d-a1f144e084c0-config\") pod \"console-operator-7cd56b8dbc-p9989\" (UID: \"068619f8-7fbb-4b22-ac7d-a1f144e084c0\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-p9989"
Nov 04 03:16:58.957530 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:58.957200    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d731646d-23ab-42d7-ae48-974929b3d2dc-networking-console-plugin-cert podName:d731646d-23ab-42d7-ae48-974929b3d2dc nodeName:}" failed. No retries permitted until 2026-11-04 03:16:59.457192078 +0000 UTC m=+34.286164603 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d731646d-23ab-42d7-ae48-974929b3d2dc-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-tlhng" (UID: "d731646d-23ab-42d7-ae48-974929b3d2dc") : secret "networking-console-plugin-cert" not found
Nov 04 03:16:58.957530 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.957217    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fc612506-b699-44bc-a81b-06fee4e819fe-installation-pull-secrets\") pod \"image-registry-7cb9f7c89d-wnpll\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") " pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:16:58.957530 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.957248    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1b21c41-4b51-4da8-8920-c38cf49554bf-metrics-certs\") pod \"router-default-7cfb798487-kbfzk\" (UID: \"c1b21c41-4b51-4da8-8920-c38cf49554bf\") " pod="openshift-ingress/router-default-7cfb798487-kbfzk"
Nov 04 03:16:58.957530 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.957269    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnrbg\" (UniqueName: \"kubernetes.io/projected/cc469246-2293-4301-9e46-bf9d609be964-kube-api-access-cnrbg\") pod \"network-check-source-6f78d59f99-7lkrx\" (UID: \"cc469246-2293-4301-9e46-bf9d609be964\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-7lkrx"
Nov 04 03:16:58.957530 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.957295    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f1f61b42-3b07-4892-90a7-2efa024e435b-ca-trust-extracted\") pod \"image-registry-79d6dd87ff-n2jnc\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") " pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:16:58.957530 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.957322    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c1b21c41-4b51-4da8-8920-c38cf49554bf-default-certificate\") pod \"router-default-7cfb798487-kbfzk\" (UID: \"c1b21c41-4b51-4da8-8920-c38cf49554bf\") " pod="openshift-ingress/router-default-7cfb798487-kbfzk"
Nov 04 03:16:58.957530 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.957343    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjtnj\" (UniqueName: \"kubernetes.io/projected/f1f61b42-3b07-4892-90a7-2efa024e435b-kube-api-access-kjtnj\") pod \"image-registry-79d6dd87ff-n2jnc\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") " pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:16:58.957530 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.957366    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b178ce9-e140-4420-81f1-627de00b90c6-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-hsb49\" (UID: \"3b178ce9-e140-4420-81f1-627de00b90c6\") " pod="openshift-insights/insights-operator-7c4986d449-hsb49"
Nov 04 03:16:58.958942 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.957388    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcm4d\" (UniqueName: \"kubernetes.io/projected/068619f8-7fbb-4b22-ac7d-a1f144e084c0-kube-api-access-mcm4d\") pod \"console-operator-7cd56b8dbc-p9989\" (UID: \"068619f8-7fbb-4b22-ac7d-a1f144e084c0\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-p9989"
Nov 04 03:16:58.958942 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.957445    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mbsz\" (UniqueName: \"kubernetes.io/projected/6dedccdc-aea2-4664-8480-ca3c9359437d-kube-api-access-5mbsz\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-dln2k\" (UID: \"6dedccdc-aea2-4664-8480-ca3c9359437d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dln2k"
Nov 04 03:16:58.958942 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.957470    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4sqp\" (UniqueName: \"kubernetes.io/projected/06698c23-887e-4ee5-9728-04098200c13d-kube-api-access-v4sqp\") pod \"dns-default-s4852\" (UID: \"06698c23-887e-4ee5-9728-04098200c13d\") " pod="openshift-dns/dns-default-s4852"
Nov 04 03:16:58.958942 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.957494    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6dedccdc-aea2-4664-8480-ca3c9359437d-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-dln2k\" (UID: \"6dedccdc-aea2-4664-8480-ca3c9359437d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dln2k"
Nov 04 03:16:58.958942 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.957523    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c1b21c41-4b51-4da8-8920-c38cf49554bf-stats-auth\") pod \"router-default-7cfb798487-kbfzk\" (UID: \"c1b21c41-4b51-4da8-8920-c38cf49554bf\") " pod="openshift-ingress/router-default-7cfb798487-kbfzk"
Nov 04 03:16:58.958942 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.957548    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghplp\" (UniqueName: \"kubernetes.io/projected/3b178ce9-e140-4420-81f1-627de00b90c6-kube-api-access-ghplp\") pod \"insights-operator-7c4986d449-hsb49\" (UID: \"3b178ce9-e140-4420-81f1-627de00b90c6\") " pod="openshift-insights/insights-operator-7c4986d449-hsb49"
Nov 04 03:16:58.958942 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.957585    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3b178ce9-e140-4420-81f1-627de00b90c6-snapshots\") pod \"insights-operator-7c4986d449-hsb49\" (UID: \"3b178ce9-e140-4420-81f1-627de00b90c6\") " pod="openshift-insights/insights-operator-7c4986d449-hsb49"
Nov 04 03:16:58.958942 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.957618    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fc612506-b699-44bc-a81b-06fee4e819fe-image-registry-private-configuration\") pod \"image-registry-7cb9f7c89d-wnpll\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") " pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:16:58.958942 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.957644    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f1f61b42-3b07-4892-90a7-2efa024e435b-trusted-ca\") pod \"image-registry-79d6dd87ff-n2jnc\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") " pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:16:58.958942 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.957673    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfc98\" (UniqueName: \"kubernetes.io/projected/fc612506-b699-44bc-a81b-06fee4e819fe-kube-api-access-bfc98\") pod \"image-registry-7cb9f7c89d-wnpll\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") " pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:16:58.958942 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.957698    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f1f61b42-3b07-4892-90a7-2efa024e435b-registry-certificates\") pod \"image-registry-79d6dd87ff-n2jnc\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") " pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:16:58.958942 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.957701    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fc612506-b699-44bc-a81b-06fee4e819fe-registry-certificates\") pod \"image-registry-7cb9f7c89d-wnpll\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") " pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:16:58.958942 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.957721    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b178ce9-e140-4420-81f1-627de00b90c6-service-ca-bundle\") pod \"insights-operator-7c4986d449-hsb49\" (UID: \"3b178ce9-e140-4420-81f1-627de00b90c6\") " pod="openshift-insights/insights-operator-7c4986d449-hsb49"
Nov 04 03:16:58.958942 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.957757    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/068619f8-7fbb-4b22-ac7d-a1f144e084c0-serving-cert\") pod \"console-operator-7cd56b8dbc-p9989\" (UID: \"068619f8-7fbb-4b22-ac7d-a1f144e084c0\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-p9989"
Nov 04 03:16:58.958942 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.957796    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/06698c23-887e-4ee5-9728-04098200c13d-metrics-tls\") pod \"dns-default-s4852\" (UID: \"06698c23-887e-4ee5-9728-04098200c13d\") " pod="openshift-dns/dns-default-s4852"
Nov 04 03:16:58.958942 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.957826    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3b178ce9-e140-4420-81f1-627de00b90c6-serving-cert\") pod \"insights-operator-7c4986d449-hsb49\" (UID: \"3b178ce9-e140-4420-81f1-627de00b90c6\") " pod="openshift-insights/insights-operator-7c4986d449-hsb49"
Nov 04 03:16:58.959738 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.957854    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/e487dae4-8ac9-4acd-a8ad-a0d149909475-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-4z8fp\" (UID: \"e487dae4-8ac9-4acd-a8ad-a0d149909475\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-4z8fp"
Nov 04 03:16:58.959738 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.957861    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e487dae4-8ac9-4acd-a8ad-a0d149909475-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-4z8fp\" (UID: \"e487dae4-8ac9-4acd-a8ad-a0d149909475\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-4z8fp"
Nov 04 03:16:58.959738 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:58.957390    2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 04 03:16:58.959738 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.957902    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f1f61b42-3b07-4892-90a7-2efa024e435b-installation-pull-secrets\") pod \"image-registry-79d6dd87ff-n2jnc\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") " pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:16:58.959738 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:58.957934    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 04 03:16:58.959738 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.958222    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b178ce9-e140-4420-81f1-627de00b90c6-service-ca-bundle\") pod \"insights-operator-7c4986d449-hsb49\" (UID: \"3b178ce9-e140-4420-81f1-627de00b90c6\") " pod="openshift-insights/insights-operator-7c4986d449-hsb49"
Nov 04 03:16:58.959738 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.958600    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6dedccdc-aea2-4664-8480-ca3c9359437d-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-dln2k\" (UID: \"6dedccdc-aea2-4664-8480-ca3c9359437d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dln2k"
Nov 04 03:16:58.959738 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:58.958838    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 03:16:58.959738 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.958901    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3b178ce9-e140-4420-81f1-627de00b90c6-snapshots\") pod \"insights-operator-7c4986d449-hsb49\" (UID: \"3b178ce9-e140-4420-81f1-627de00b90c6\") " pod="openshift-insights/insights-operator-7c4986d449-hsb49"
Nov 04 03:16:58.959738 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.959703    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/068619f8-7fbb-4b22-ac7d-a1f144e084c0-trusted-ca\") pod \"console-operator-7cd56b8dbc-p9989\" (UID: \"068619f8-7fbb-4b22-ac7d-a1f144e084c0\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-p9989"
Nov 04 03:16:58.960193 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.959937    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b178ce9-e140-4420-81f1-627de00b90c6-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-hsb49\" (UID: \"3b178ce9-e140-4420-81f1-627de00b90c6\") " pod="openshift-insights/insights-operator-7c4986d449-hsb49"
Nov 04 03:16:58.960193 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:58.957938    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1b21c41-4b51-4da8-8920-c38cf49554bf-metrics-certs podName:c1b21c41-4b51-4da8-8920-c38cf49554bf nodeName:}" failed. No retries permitted until 2026-11-04 03:16:59.45792466 +0000 UTC m=+34.286897191 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c1b21c41-4b51-4da8-8920-c38cf49554bf-metrics-certs") pod "router-default-7cfb798487-kbfzk" (UID: "c1b21c41-4b51-4da8-8920-c38cf49554bf") : secret "router-metrics-certs-default" not found
Nov 04 03:16:58.960193 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.959990    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d731646d-23ab-42d7-ae48-974929b3d2dc-nginx-conf\") pod \"networking-console-plugin-86c478bb88-tlhng\" (UID: \"d731646d-23ab-42d7-ae48-974929b3d2dc\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-tlhng"
Nov 04 03:16:58.960193 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.960025    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fc612506-b699-44bc-a81b-06fee4e819fe-ca-trust-extracted\") pod \"image-registry-7cb9f7c89d-wnpll\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") " pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:16:58.960193 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.960055    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlxxz\" (UniqueName: \"kubernetes.io/projected/e487dae4-8ac9-4acd-a8ad-a0d149909475-kube-api-access-vlxxz\") pod \"cluster-samples-operator-6ddb6c8b94-4z8fp\" (UID: \"e487dae4-8ac9-4acd-a8ad-a0d149909475\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-4z8fp"
Nov 04 03:16:58.960193 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.960082    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f1f61b42-3b07-4892-90a7-2efa024e435b-registry-tls\") pod \"image-registry-79d6dd87ff-n2jnc\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") " pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:16:58.960193 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.960118    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f1f61b42-3b07-4892-90a7-2efa024e435b-bound-sa-token\") pod \"image-registry-79d6dd87ff-n2jnc\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") " pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:16:58.960193 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.960146    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6dedccdc-aea2-4664-8480-ca3c9359437d-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-dln2k\" (UID: \"6dedccdc-aea2-4664-8480-ca3c9359437d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dln2k"
Nov 04 03:16:58.960193 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.960173    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0beb091e-5c2f-4136-93a2-771f9b861411-serving-cert\") pod \"service-ca-operator-544d749c8c-js5xd\" (UID: \"0beb091e-5c2f-4136-93a2-771f9b861411\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-js5xd"
Nov 04 03:16:58.961911 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.960771    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fc612506-b699-44bc-a81b-06fee4e819fe-ca-trust-extracted\") pod \"image-registry-7cb9f7c89d-wnpll\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") " pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:16:58.961911 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.961147    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f1f61b42-3b07-4892-90a7-2efa024e435b-image-registry-private-configuration\") pod \"image-registry-79d6dd87ff-n2jnc\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") " pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:16:58.961911 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:58.961194    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e487dae4-8ac9-4acd-a8ad-a0d149909475-samples-operator-tls podName:e487dae4-8ac9-4acd-a8ad-a0d149909475 nodeName:}" failed. No retries permitted until 2026-11-04 03:16:59.461179414 +0000 UTC m=+34.290151944 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e487dae4-8ac9-4acd-a8ad-a0d149909475-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-4z8fp" (UID: "e487dae4-8ac9-4acd-a8ad-a0d149909475") : secret "samples-operator-tls" not found
Nov 04 03:16:58.961911 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:58.961214    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06698c23-887e-4ee5-9728-04098200c13d-metrics-tls podName:06698c23-887e-4ee5-9728-04098200c13d nodeName:}" failed. No retries permitted until 2026-11-04 03:16:59.461205984 +0000 UTC m=+34.290178500 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/06698c23-887e-4ee5-9728-04098200c13d-metrics-tls") pod "dns-default-s4852" (UID: "06698c23-887e-4ee5-9728-04098200c13d") : secret "dns-default-metrics-tls" not found
Nov 04 03:16:58.961911 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.961433    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f1f61b42-3b07-4892-90a7-2efa024e435b-registry-certificates\") pod \"image-registry-79d6dd87ff-n2jnc\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") " pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:16:58.961911 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:58.961534    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 03:16:58.961911 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:58.961547    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79d6dd87ff-n2jnc: secret "image-registry-tls" not found
Nov 04 03:16:58.961911 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.961556    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fc612506-b699-44bc-a81b-06fee4e819fe-installation-pull-secrets\") pod \"image-registry-7cb9f7c89d-wnpll\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") " pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:16:58.961911 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:58.961589    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f1f61b42-3b07-4892-90a7-2efa024e435b-registry-tls podName:f1f61b42-3b07-4892-90a7-2efa024e435b nodeName:}" failed. No retries permitted until 2026-11-04 03:16:59.461575307 +0000 UTC m=+34.290547832 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f1f61b42-3b07-4892-90a7-2efa024e435b-registry-tls") pod "image-registry-79d6dd87ff-n2jnc" (UID: "f1f61b42-3b07-4892-90a7-2efa024e435b") : secret "image-registry-tls" not found
Nov 04 03:16:58.961911 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.961599    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d731646d-23ab-42d7-ae48-974929b3d2dc-nginx-conf\") pod \"networking-console-plugin-86c478bb88-tlhng\" (UID: \"d731646d-23ab-42d7-ae48-974929b3d2dc\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-tlhng"
Nov 04 03:16:58.962462 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.962024    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f1f61b42-3b07-4892-90a7-2efa024e435b-ca-trust-extracted\") pod \"image-registry-79d6dd87ff-n2jnc\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") " pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:16:58.962462 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.962079    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f1f61b42-3b07-4892-90a7-2efa024e435b-trusted-ca\") pod \"image-registry-79d6dd87ff-n2jnc\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") " pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:16:58.963325 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.963115    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/068619f8-7fbb-4b22-ac7d-a1f144e084c0-serving-cert\") pod \"console-operator-7cd56b8dbc-p9989\" (UID: \"068619f8-7fbb-4b22-ac7d-a1f144e084c0\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-p9989"
Nov 04 03:16:58.963325 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.963251    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3b178ce9-e140-4420-81f1-627de00b90c6-serving-cert\") pod \"insights-operator-7c4986d449-hsb49\" (UID: \"3b178ce9-e140-4420-81f1-627de00b90c6\") " pod="openshift-insights/insights-operator-7c4986d449-hsb49"
Nov 04 03:16:58.963325 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.963283    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fc612506-b699-44bc-a81b-06fee4e819fe-image-registry-private-configuration\") pod \"image-registry-7cb9f7c89d-wnpll\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") " pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:16:58.963612 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.963442    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/55c4af4c-188b-495b-9052-5700a1f60602-cert\") pod \"ingress-canary-qbskg\" (UID: \"55c4af4c-188b-495b-9052-5700a1f60602\") " pod="openshift-ingress-canary/ingress-canary-qbskg"
Nov 04 03:16:58.963716 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.963691    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c1b21c41-4b51-4da8-8920-c38cf49554bf-default-certificate\") pod \"router-default-7cfb798487-kbfzk\" (UID: \"c1b21c41-4b51-4da8-8920-c38cf49554bf\") " pod="openshift-ingress/router-default-7cfb798487-kbfzk"
Nov 04 03:16:58.963844 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.963765    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c1b21c41-4b51-4da8-8920-c38cf49554bf-stats-auth\") pod \"router-default-7cfb798487-kbfzk\" (UID: \"c1b21c41-4b51-4da8-8920-c38cf49554bf\") " pod="openshift-ingress/router-default-7cfb798487-kbfzk"
Nov 04 03:16:58.963953 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.963930    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fc612506-b699-44bc-a81b-06fee4e819fe-trusted-ca\") pod \"image-registry-7cb9f7c89d-wnpll\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") " pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:16:58.964564 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.964545    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0beb091e-5c2f-4136-93a2-771f9b861411-serving-cert\") pod \"service-ca-operator-544d749c8c-js5xd\" (UID: \"0beb091e-5c2f-4136-93a2-771f9b861411\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-js5xd"
Nov 04 03:16:58.964701 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.964675    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f1f61b42-3b07-4892-90a7-2efa024e435b-installation-pull-secrets\") pod \"image-registry-79d6dd87ff-n2jnc\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") " pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:16:58.965233 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.965216    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6dedccdc-aea2-4664-8480-ca3c9359437d-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-dln2k\" (UID: \"6dedccdc-aea2-4664-8480-ca3c9359437d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dln2k"
Nov 04 03:16:58.976444 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.976395    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-n6r7s"
Nov 04 03:16:58.984539 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:58.984378    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-mdchx"
Nov 04 03:16:59.021336 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.021056    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5h469\" (UniqueName: \"kubernetes.io/projected/55c4af4c-188b-495b-9052-5700a1f60602-kube-api-access-5h469\") pod \"ingress-canary-qbskg\" (UID: \"55c4af4c-188b-495b-9052-5700a1f60602\") " pod="openshift-ingress-canary/ingress-canary-qbskg"
Nov 04 03:16:59.023912 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.023862    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjtnj\" (UniqueName: \"kubernetes.io/projected/f1f61b42-3b07-4892-90a7-2efa024e435b-kube-api-access-kjtnj\") pod \"image-registry-79d6dd87ff-n2jnc\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") " pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:16:59.026056 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.025354    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4pj6\" (UniqueName: \"kubernetes.io/projected/0beb091e-5c2f-4136-93a2-771f9b861411-kube-api-access-m4pj6\") pod \"service-ca-operator-544d749c8c-js5xd\" (UID: \"0beb091e-5c2f-4136-93a2-771f9b861411\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-js5xd"
Nov 04 03:16:59.026056 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.025549    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsjms\" (UniqueName: \"kubernetes.io/projected/c1b21c41-4b51-4da8-8920-c38cf49554bf-kube-api-access-fsjms\") pod \"router-default-7cfb798487-kbfzk\" (UID: \"c1b21c41-4b51-4da8-8920-c38cf49554bf\") " pod="openshift-ingress/router-default-7cfb798487-kbfzk"
Nov 04 03:16:59.026056 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.025889    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfc98\" (UniqueName: \"kubernetes.io/projected/fc612506-b699-44bc-a81b-06fee4e819fe-kube-api-access-bfc98\") pod \"image-registry-7cb9f7c89d-wnpll\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") " pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:16:59.026056 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.025925    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f1f61b42-3b07-4892-90a7-2efa024e435b-bound-sa-token\") pod \"image-registry-79d6dd87ff-n2jnc\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") " pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:16:59.026056 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.026006    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8g8qf\" (UniqueName: \"kubernetes.io/projected/e451b329-bd05-44f5-9544-8283f76df9d6-kube-api-access-8g8qf\") pod \"volume-data-source-validator-58c9f5f46d-6c2jp\" (UID: \"e451b329-bd05-44f5-9544-8283f76df9d6\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-6c2jp"
Nov 04 03:16:59.026391 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.026153    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mbsz\" (UniqueName: \"kubernetes.io/projected/6dedccdc-aea2-4664-8480-ca3c9359437d-kube-api-access-5mbsz\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-dln2k\" (UID: \"6dedccdc-aea2-4664-8480-ca3c9359437d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dln2k"
Nov 04 03:16:59.027113 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.027089    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbrl5\" (UniqueName: \"kubernetes.io/projected/e87dded5-1e1b-45cb-ab2e-ae12550dfca7-kube-api-access-sbrl5\") pod \"cluster-monitoring-operator-884576c57-gmt9g\" (UID: \"e87dded5-1e1b-45cb-ab2e-ae12550dfca7\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmt9g"
Nov 04 03:16:59.027204 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.027144    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlxxz\" (UniqueName: \"kubernetes.io/projected/e487dae4-8ac9-4acd-a8ad-a0d149909475-kube-api-access-vlxxz\") pod \"cluster-samples-operator-6ddb6c8b94-4z8fp\" (UID: \"e487dae4-8ac9-4acd-a8ad-a0d149909475\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-4z8fp"
Nov 04 03:16:59.027682 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.027622    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcm4d\" (UniqueName: \"kubernetes.io/projected/068619f8-7fbb-4b22-ac7d-a1f144e084c0-kube-api-access-mcm4d\") pod \"console-operator-7cd56b8dbc-p9989\" (UID: \"068619f8-7fbb-4b22-ac7d-a1f144e084c0\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-p9989"
Nov 04 03:16:59.027826 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.027796    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnrbg\" (UniqueName: \"kubernetes.io/projected/cc469246-2293-4301-9e46-bf9d609be964-kube-api-access-cnrbg\") pod \"network-check-source-6f78d59f99-7lkrx\" (UID: \"cc469246-2293-4301-9e46-bf9d609be964\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-7lkrx"
Nov 04 03:16:59.031487 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.028470    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghplp\" (UniqueName: \"kubernetes.io/projected/3b178ce9-e140-4420-81f1-627de00b90c6-kube-api-access-ghplp\") pod \"insights-operator-7c4986d449-hsb49\" (UID: \"3b178ce9-e140-4420-81f1-627de00b90c6\") " pod="openshift-insights/insights-operator-7c4986d449-hsb49"
Nov 04 03:16:59.031487 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.028576    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4sqp\" (UniqueName: \"kubernetes.io/projected/06698c23-887e-4ee5-9728-04098200c13d-kube-api-access-v4sqp\") pod \"dns-default-s4852\" (UID: \"06698c23-887e-4ee5-9728-04098200c13d\") " pod="openshift-dns/dns-default-s4852"
Nov 04 03:16:59.033532 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.032871    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fc612506-b699-44bc-a81b-06fee4e819fe-bound-sa-token\") pod \"image-registry-7cb9f7c89d-wnpll\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") " pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:16:59.050430 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.050322    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-p9989"
Nov 04 03:16:59.079659 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.079630    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-hsb49"
Nov 04 03:16:59.089817 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.089796    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-7lkrx"
Nov 04 03:16:59.101748 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.101726    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qbskg"
Nov 04 03:16:59.120659 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.117390    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hjssc"
Nov 04 03:16:59.290141 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.290104    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-js5xd"
Nov 04 03:16:59.303338 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.303307    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dln2k"
Nov 04 03:16:59.317713 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.317690    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-xpmwt"]
Nov 04 03:16:59.318862 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.318840    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-6c2jp"
Nov 04 03:16:59.326577 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:59.326550    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda51f26a7_0ec8_4c56_881a_98c6d625478b.slice/crio-f0f6858bbc07364162f2e8e9791a42974915aaf66d257711b085ec12ad5badf8 WatchSource:0}: Error finding container f0f6858bbc07364162f2e8e9791a42974915aaf66d257711b085ec12ad5badf8: Status 404 returned error can't find the container with id f0f6858bbc07364162f2e8e9791a42974915aaf66d257711b085ec12ad5badf8
Nov 04 03:16:59.363939 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.363812    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-n6r7s"]
Nov 04 03:16:59.366154 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.365651    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-mdchx"]
Nov 04 03:16:59.383010 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:59.382738    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0c72601_2624_4632_9f15_c19fcc489aae.slice/crio-9e95f53696261300915c372199df2f62ecad13630d51184d45c933312e5b9ca1 WatchSource:0}: Error finding container 9e95f53696261300915c372199df2f62ecad13630d51184d45c933312e5b9ca1: Status 404 returned error can't find the container with id 9e95f53696261300915c372199df2f62ecad13630d51184d45c933312e5b9ca1
Nov 04 03:16:59.383376 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:59.383347    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10cd06d4_ef1a_4ecf_a72c_e6d0084f3032.slice/crio-6736a5a06164268694d3b79c2102d16b063d5d57f873ea08e2c53ffcad7f08c7 WatchSource:0}: Error finding container 6736a5a06164268694d3b79c2102d16b063d5d57f873ea08e2c53ffcad7f08c7: Status 404 returned error can't find the container with id 6736a5a06164268694d3b79c2102d16b063d5d57f873ea08e2c53ffcad7f08c7
Nov 04 03:16:59.460725 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.460684    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-hsb49"]
Nov 04 03:16:59.470961 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.470931    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1b21c41-4b51-4da8-8920-c38cf49554bf-metrics-certs\") pod \"router-default-7cfb798487-kbfzk\" (UID: \"c1b21c41-4b51-4da8-8920-c38cf49554bf\") " pod="openshift-ingress/router-default-7cfb798487-kbfzk"
Nov 04 03:16:59.471066 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.471008    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/06698c23-887e-4ee5-9728-04098200c13d-metrics-tls\") pod \"dns-default-s4852\" (UID: \"06698c23-887e-4ee5-9728-04098200c13d\") " pod="openshift-dns/dns-default-s4852"
Nov 04 03:16:59.471137 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:59.471121    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 03:16:59.471192 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:59.471134    2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 04 03:16:59.471238 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.471185    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e487dae4-8ac9-4acd-a8ad-a0d149909475-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-4z8fp\" (UID: \"e487dae4-8ac9-4acd-a8ad-a0d149909475\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-4z8fp"
Nov 04 03:16:59.471238 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:59.471211    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1b21c41-4b51-4da8-8920-c38cf49554bf-metrics-certs podName:c1b21c41-4b51-4da8-8920-c38cf49554bf nodeName:}" failed. No retries permitted until 2026-11-04 03:17:00.471191352 +0000 UTC m=+35.300163863 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c1b21c41-4b51-4da8-8920-c38cf49554bf-metrics-certs") pod "router-default-7cfb798487-kbfzk" (UID: "c1b21c41-4b51-4da8-8920-c38cf49554bf") : secret "router-metrics-certs-default" not found
Nov 04 03:16:59.471344 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:59.471241    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06698c23-887e-4ee5-9728-04098200c13d-metrics-tls podName:06698c23-887e-4ee5-9728-04098200c13d nodeName:}" failed. No retries permitted until 2026-11-04 03:17:00.4712324 +0000 UTC m=+35.300204912 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/06698c23-887e-4ee5-9728-04098200c13d-metrics-tls") pod "dns-default-s4852" (UID: "06698c23-887e-4ee5-9728-04098200c13d") : secret "dns-default-metrics-tls" not found
Nov 04 03:16:59.471344 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:59.471268    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 04 03:16:59.471344 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.471281    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f1f61b42-3b07-4892-90a7-2efa024e435b-registry-tls\") pod \"image-registry-79d6dd87ff-n2jnc\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") " pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:16:59.471344 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.471338    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e87dded5-1e1b-45cb-ab2e-ae12550dfca7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-gmt9g\" (UID: \"e87dded5-1e1b-45cb-ab2e-ae12550dfca7\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmt9g"
Nov 04 03:16:59.471555 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:59.471354    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e487dae4-8ac9-4acd-a8ad-a0d149909475-samples-operator-tls podName:e487dae4-8ac9-4acd-a8ad-a0d149909475 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:00.471339481 +0000 UTC m=+35.300311995 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e487dae4-8ac9-4acd-a8ad-a0d149909475-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-4z8fp" (UID: "e487dae4-8ac9-4acd-a8ad-a0d149909475") : secret "samples-operator-tls" not found
Nov 04 03:16:59.471555 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:59.471419    2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 04 03:16:59.471555 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.471419    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1b21c41-4b51-4da8-8920-c38cf49554bf-service-ca-bundle\") pod \"router-default-7cfb798487-kbfzk\" (UID: \"c1b21c41-4b51-4da8-8920-c38cf49554bf\") " pod="openshift-ingress/router-default-7cfb798487-kbfzk"
Nov 04 03:16:59.471555 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:59.471457    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e87dded5-1e1b-45cb-ab2e-ae12550dfca7-cluster-monitoring-operator-tls podName:e87dded5-1e1b-45cb-ab2e-ae12550dfca7 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:00.471445313 +0000 UTC m=+35.300417825 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e87dded5-1e1b-45cb-ab2e-ae12550dfca7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-gmt9g" (UID: "e87dded5-1e1b-45cb-ab2e-ae12550dfca7") : secret "cluster-monitoring-operator-tls" not found
Nov 04 03:16:59.471555 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:59.471468    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 03:16:59.471555 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.471474    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fc612506-b699-44bc-a81b-06fee4e819fe-registry-tls\") pod \"image-registry-7cb9f7c89d-wnpll\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") " pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:16:59.471555 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:59.471483    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79d6dd87ff-n2jnc: secret "image-registry-tls" not found
Nov 04 03:16:59.471555 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:59.471505    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c1b21c41-4b51-4da8-8920-c38cf49554bf-service-ca-bundle podName:c1b21c41-4b51-4da8-8920-c38cf49554bf nodeName:}" failed. No retries permitted until 2026-11-04 03:17:00.471494991 +0000 UTC m=+35.300467556 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c1b21c41-4b51-4da8-8920-c38cf49554bf-service-ca-bundle") pod "router-default-7cfb798487-kbfzk" (UID: "c1b21c41-4b51-4da8-8920-c38cf49554bf") : configmap references non-existent config key: service-ca.crt
Nov 04 03:16:59.471555 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:59.471529    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f1f61b42-3b07-4892-90a7-2efa024e435b-registry-tls podName:f1f61b42-3b07-4892-90a7-2efa024e435b nodeName:}" failed. No retries permitted until 2026-11-04 03:17:00.471514603 +0000 UTC m=+35.300487127 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f1f61b42-3b07-4892-90a7-2efa024e435b-registry-tls") pod "image-registry-79d6dd87ff-n2jnc" (UID: "f1f61b42-3b07-4892-90a7-2efa024e435b") : secret "image-registry-tls" not found
Nov 04 03:16:59.471555 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:59.471547    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 03:16:59.471555 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:59.471558    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cb9f7c89d-wnpll: secret "image-registry-tls" not found
Nov 04 03:16:59.472043 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.471568    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d731646d-23ab-42d7-ae48-974929b3d2dc-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-tlhng\" (UID: \"d731646d-23ab-42d7-ae48-974929b3d2dc\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-tlhng"
Nov 04 03:16:59.472043 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:59.471600    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fc612506-b699-44bc-a81b-06fee4e819fe-registry-tls podName:fc612506-b699-44bc-a81b-06fee4e819fe nodeName:}" failed. No retries permitted until 2026-11-04 03:17:00.471590172 +0000 UTC m=+35.300562685 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fc612506-b699-44bc-a81b-06fee4e819fe-registry-tls") pod "image-registry-7cb9f7c89d-wnpll" (UID: "fc612506-b699-44bc-a81b-06fee4e819fe") : secret "image-registry-tls" not found
Nov 04 03:16:59.472043 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:59.471666    2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 04 03:16:59.472043 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:16:59.471703    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d731646d-23ab-42d7-ae48-974929b3d2dc-networking-console-plugin-cert podName:d731646d-23ab-42d7-ae48-974929b3d2dc nodeName:}" failed. No retries permitted until 2026-11-04 03:17:00.471691452 +0000 UTC m=+35.300663983 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d731646d-23ab-42d7-ae48-974929b3d2dc-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-tlhng" (UID: "d731646d-23ab-42d7-ae48-974929b3d2dc") : secret "networking-console-plugin-cert" not found
Nov 04 03:16:59.475292 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.475270    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-7lkrx"]
Nov 04 03:16:59.489153 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:59.489076    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc469246_2293_4301_9e46_bf9d609be964.slice/crio-fcdc4285e3406fb4f6124f6289bba3ee08d73e97bbd67a8a91fa342b72c84c10 WatchSource:0}: Error finding container fcdc4285e3406fb4f6124f6289bba3ee08d73e97bbd67a8a91fa342b72c84c10: Status 404 returned error can't find the container with id fcdc4285e3406fb4f6124f6289bba3ee08d73e97bbd67a8a91fa342b72c84c10
Nov 04 03:16:59.501371 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.501290    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qbskg"]
Nov 04 03:16:59.507795 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.507504    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hjssc"]
Nov 04 03:16:59.511662 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:59.510367    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55c4af4c_188b_495b_9052_5700a1f60602.slice/crio-ff6df41ba86f2e45d61b592fca5e1d28949c7c919df146f3ce07e1d0856c3a2a WatchSource:0}: Error finding container ff6df41ba86f2e45d61b592fca5e1d28949c7c919df146f3ce07e1d0856c3a2a: Status 404 returned error can't find the container with id ff6df41ba86f2e45d61b592fca5e1d28949c7c919df146f3ce07e1d0856c3a2a
Nov 04 03:16:59.521079 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:59.521047    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04b955ef_528b_4573_b18f_0957658f79e4.slice/crio-8da439f9f72d07468b5f026f679257e73c6e569c195c803db1bcec906507c8b3 WatchSource:0}: Error finding container 8da439f9f72d07468b5f026f679257e73c6e569c195c803db1bcec906507c8b3: Status 404 returned error can't find the container with id 8da439f9f72d07468b5f026f679257e73c6e569c195c803db1bcec906507c8b3
Nov 04 03:16:59.547831 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.547441    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-p9989"]
Nov 04 03:16:59.555662 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:59.555637    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod068619f8_7fbb_4b22_ac7d_a1f144e084c0.slice/crio-4ef5544e200695af7b630a18f1d5f1c548e3f5c6b5776ade65da67c07f469ee7 WatchSource:0}: Error finding container 4ef5544e200695af7b630a18f1d5f1c548e3f5c6b5776ade65da67c07f469ee7: Status 404 returned error can't find the container with id 4ef5544e200695af7b630a18f1d5f1c548e3f5c6b5776ade65da67c07f469ee7
Nov 04 03:16:59.662245 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.662131    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-js5xd"]
Nov 04 03:16:59.668533 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:59.668502    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0beb091e_5c2f_4136_93a2_771f9b861411.slice/crio-2213f8bdf00825959d18e17459271b780f1b89136caac2eec3a8cd654924888b WatchSource:0}: Error finding container 2213f8bdf00825959d18e17459271b780f1b89136caac2eec3a8cd654924888b: Status 404 returned error can't find the container with id 2213f8bdf00825959d18e17459271b780f1b89136caac2eec3a8cd654924888b
Nov 04 03:16:59.693197 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.693168    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dln2k"]
Nov 04 03:16:59.700274 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:59.700243    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dedccdc_aea2_4664_8480_ca3c9359437d.slice/crio-32351bff9b6d4c90f5736f07fe59a8cab87fca6348451508e1e7cf33b95596b2 WatchSource:0}: Error finding container 32351bff9b6d4c90f5736f07fe59a8cab87fca6348451508e1e7cf33b95596b2: Status 404 returned error can't find the container with id 32351bff9b6d4c90f5736f07fe59a8cab87fca6348451508e1e7cf33b95596b2
Nov 04 03:16:59.716266 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.716236    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-6c2jp"]
Nov 04 03:16:59.723773 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:16:59.723749    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode451b329_bd05_44f5_9544_8283f76df9d6.slice/crio-cdb60b525560e15019a37aaf257c607d28f8bb9de40e391f8e06d0f43a2a4524 WatchSource:0}: Error finding container cdb60b525560e15019a37aaf257c607d28f8bb9de40e391f8e06d0f43a2a4524: Status 404 returned error can't find the container with id cdb60b525560e15019a37aaf257c607d28f8bb9de40e391f8e06d0f43a2a4524
Nov 04 03:16:59.926515 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.926431    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-7lkrx" event={"ID":"cc469246-2293-4301-9e46-bf9d609be964","Type":"ContainerStarted","Data":"fcdc4285e3406fb4f6124f6289bba3ee08d73e97bbd67a8a91fa342b72c84c10"}
Nov 04 03:16:59.927645 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.927621    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-p9989" event={"ID":"068619f8-7fbb-4b22-ac7d-a1f144e084c0","Type":"ContainerStarted","Data":"4ef5544e200695af7b630a18f1d5f1c548e3f5c6b5776ade65da67c07f469ee7"}
Nov 04 03:16:59.928841 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.928814    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-6c2jp" event={"ID":"e451b329-bd05-44f5-9544-8283f76df9d6","Type":"ContainerStarted","Data":"cdb60b525560e15019a37aaf257c607d28f8bb9de40e391f8e06d0f43a2a4524"}
Nov 04 03:16:59.929983 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.929957    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hjssc" event={"ID":"04b955ef-528b-4573-b18f-0957658f79e4","Type":"ContainerStarted","Data":"8da439f9f72d07468b5f026f679257e73c6e569c195c803db1bcec906507c8b3"}
Nov 04 03:16:59.931617 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.931582    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-mdchx" event={"ID":"e0c72601-2624-4632-9f15-c19fcc489aae","Type":"ContainerStarted","Data":"802303c06bdd576979d923384fddd8a166d25f486369d7a3da77cd4c774a9e0f"}
Nov 04 03:16:59.931709 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.931625    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-mdchx" event={"ID":"e0c72601-2624-4632-9f15-c19fcc489aae","Type":"ContainerStarted","Data":"9e95f53696261300915c372199df2f62ecad13630d51184d45c933312e5b9ca1"}
Nov 04 03:16:59.933678 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.933650    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-n6r7s" event={"ID":"10cd06d4-ef1a-4ecf-a72c-e6d0084f3032","Type":"ContainerStarted","Data":"f5ba1513a696039f344aec559decf727b1241e65f475b46d25f974d64d0f7fb2"}
Nov 04 03:16:59.933808 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.933698    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-n6r7s" event={"ID":"10cd06d4-ef1a-4ecf-a72c-e6d0084f3032","Type":"ContainerStarted","Data":"6736a5a06164268694d3b79c2102d16b063d5d57f873ea08e2c53ffcad7f08c7"}
Nov 04 03:16:59.935229 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.935203    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-js5xd" event={"ID":"0beb091e-5c2f-4136-93a2-771f9b861411","Type":"ContainerStarted","Data":"2213f8bdf00825959d18e17459271b780f1b89136caac2eec3a8cd654924888b"}
Nov 04 03:16:59.936615 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.936593    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dln2k" event={"ID":"6dedccdc-aea2-4664-8480-ca3c9359437d","Type":"ContainerStarted","Data":"32351bff9b6d4c90f5736f07fe59a8cab87fca6348451508e1e7cf33b95596b2"}
Nov 04 03:16:59.938280 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.938236    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-xpmwt" event={"ID":"a51f26a7-0ec8-4c56-881a-98c6d625478b","Type":"ContainerStarted","Data":"e2326881575de5c68bcd67be148f412554e36c95f12055d13e55d0e540087b06"}
Nov 04 03:16:59.938381 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.938297    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-xpmwt" event={"ID":"a51f26a7-0ec8-4c56-881a-98c6d625478b","Type":"ContainerStarted","Data":"f0f6858bbc07364162f2e8e9791a42974915aaf66d257711b085ec12ad5badf8"}
Nov 04 03:16:59.939891 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.939871    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qbskg" event={"ID":"55c4af4c-188b-495b-9052-5700a1f60602","Type":"ContainerStarted","Data":"ff6df41ba86f2e45d61b592fca5e1d28949c7c919df146f3ce07e1d0856c3a2a"}
Nov 04 03:16:59.941605 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:16:59.941577    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-hsb49" event={"ID":"3b178ce9-e140-4420-81f1-627de00b90c6","Type":"ContainerStarted","Data":"e2233f4f39df1b4a827fd6740e336d1243793a92a49423b5e6d0087113b0f7fe"}
Nov 04 03:17:00.480534 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:00.480496    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/06698c23-887e-4ee5-9728-04098200c13d-metrics-tls\") pod \"dns-default-s4852\" (UID: \"06698c23-887e-4ee5-9728-04098200c13d\") " pod="openshift-dns/dns-default-s4852"
Nov 04 03:17:00.480534 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:00.480545    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e487dae4-8ac9-4acd-a8ad-a0d149909475-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-4z8fp\" (UID: \"e487dae4-8ac9-4acd-a8ad-a0d149909475\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-4z8fp"
Nov 04 03:17:00.481057 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:00.480577    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f1f61b42-3b07-4892-90a7-2efa024e435b-registry-tls\") pod \"image-registry-79d6dd87ff-n2jnc\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") " pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:17:00.481057 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:00.480613    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e87dded5-1e1b-45cb-ab2e-ae12550dfca7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-gmt9g\" (UID: \"e87dded5-1e1b-45cb-ab2e-ae12550dfca7\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmt9g"
Nov 04 03:17:00.481057 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:00.480643    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1b21c41-4b51-4da8-8920-c38cf49554bf-service-ca-bundle\") pod \"router-default-7cfb798487-kbfzk\" (UID: \"c1b21c41-4b51-4da8-8920-c38cf49554bf\") " pod="openshift-ingress/router-default-7cfb798487-kbfzk"
Nov 04 03:17:00.481057 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:00.480647    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 03:17:00.481057 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:00.480681    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fc612506-b699-44bc-a81b-06fee4e819fe-registry-tls\") pod \"image-registry-7cb9f7c89d-wnpll\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") " pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:17:00.481057 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:00.480728    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06698c23-887e-4ee5-9728-04098200c13d-metrics-tls podName:06698c23-887e-4ee5-9728-04098200c13d nodeName:}" failed. No retries permitted until 2026-11-04 03:17:02.480695259 +0000 UTC m=+37.309667770 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/06698c23-887e-4ee5-9728-04098200c13d-metrics-tls") pod "dns-default-s4852" (UID: "06698c23-887e-4ee5-9728-04098200c13d") : secret "dns-default-metrics-tls" not found
Nov 04 03:17:00.481057 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:00.480742    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 04 03:17:00.481057 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:00.480777    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 03:17:00.481057 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:00.480786    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d731646d-23ab-42d7-ae48-974929b3d2dc-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-tlhng\" (UID: \"d731646d-23ab-42d7-ae48-974929b3d2dc\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-tlhng"
Nov 04 03:17:00.481057 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:00.480791    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cb9f7c89d-wnpll: secret "image-registry-tls" not found
Nov 04 03:17:00.481057 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:00.480777    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 03:17:00.481057 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:00.480832    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79d6dd87ff-n2jnc: secret "image-registry-tls" not found
Nov 04 03:17:00.481057 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:00.480792    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e487dae4-8ac9-4acd-a8ad-a0d149909475-samples-operator-tls podName:e487dae4-8ac9-4acd-a8ad-a0d149909475 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:02.480779917 +0000 UTC m=+37.309752427 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e487dae4-8ac9-4acd-a8ad-a0d149909475-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-4z8fp" (UID: "e487dae4-8ac9-4acd-a8ad-a0d149909475") : secret "samples-operator-tls" not found
Nov 04 03:17:00.481057 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:00.480847    2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 04 03:17:00.481057 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:00.480860    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fc612506-b699-44bc-a81b-06fee4e819fe-registry-tls podName:fc612506-b699-44bc-a81b-06fee4e819fe nodeName:}" failed. No retries permitted until 2026-11-04 03:17:02.48085325 +0000 UTC m=+37.309825760 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fc612506-b699-44bc-a81b-06fee4e819fe-registry-tls") pod "image-registry-7cb9f7c89d-wnpll" (UID: "fc612506-b699-44bc-a81b-06fee4e819fe") : secret "image-registry-tls" not found
Nov 04 03:17:00.481057 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:00.480872    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c1b21c41-4b51-4da8-8920-c38cf49554bf-service-ca-bundle podName:c1b21c41-4b51-4da8-8920-c38cf49554bf nodeName:}" failed. No retries permitted until 2026-11-04 03:17:02.480866681 +0000 UTC m=+37.309839191 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c1b21c41-4b51-4da8-8920-c38cf49554bf-service-ca-bundle") pod "router-default-7cfb798487-kbfzk" (UID: "c1b21c41-4b51-4da8-8920-c38cf49554bf") : configmap references non-existent config key: service-ca.crt
Nov 04 03:17:00.481057 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:00.480847    2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 04 03:17:00.481611 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:00.480888    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f1f61b42-3b07-4892-90a7-2efa024e435b-registry-tls podName:f1f61b42-3b07-4892-90a7-2efa024e435b nodeName:}" failed. No retries permitted until 2026-11-04 03:17:02.48088019 +0000 UTC m=+37.309852700 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f1f61b42-3b07-4892-90a7-2efa024e435b-registry-tls") pod "image-registry-79d6dd87ff-n2jnc" (UID: "f1f61b42-3b07-4892-90a7-2efa024e435b") : secret "image-registry-tls" not found
Nov 04 03:17:00.481611 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:00.480905    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d731646d-23ab-42d7-ae48-974929b3d2dc-networking-console-plugin-cert podName:d731646d-23ab-42d7-ae48-974929b3d2dc nodeName:}" failed. No retries permitted until 2026-11-04 03:17:02.480899285 +0000 UTC m=+37.309871796 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d731646d-23ab-42d7-ae48-974929b3d2dc-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-tlhng" (UID: "d731646d-23ab-42d7-ae48-974929b3d2dc") : secret "networking-console-plugin-cert" not found
Nov 04 03:17:00.481611 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:00.480917    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e87dded5-1e1b-45cb-ab2e-ae12550dfca7-cluster-monitoring-operator-tls podName:e87dded5-1e1b-45cb-ab2e-ae12550dfca7 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:02.480912566 +0000 UTC m=+37.309885075 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e87dded5-1e1b-45cb-ab2e-ae12550dfca7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-gmt9g" (UID: "e87dded5-1e1b-45cb-ab2e-ae12550dfca7") : secret "cluster-monitoring-operator-tls" not found
Nov 04 03:17:00.481611 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:00.480957    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1b21c41-4b51-4da8-8920-c38cf49554bf-metrics-certs\") pod \"router-default-7cfb798487-kbfzk\" (UID: \"c1b21c41-4b51-4da8-8920-c38cf49554bf\") " pod="openshift-ingress/router-default-7cfb798487-kbfzk"
Nov 04 03:17:00.481611 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:00.481079    2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 04 03:17:00.481611 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:00.481122    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1b21c41-4b51-4da8-8920-c38cf49554bf-metrics-certs podName:c1b21c41-4b51-4da8-8920-c38cf49554bf nodeName:}" failed. No retries permitted until 2026-11-04 03:17:02.481106669 +0000 UTC m=+37.310079181 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c1b21c41-4b51-4da8-8920-c38cf49554bf-metrics-certs") pod "router-default-7cfb798487-kbfzk" (UID: "c1b21c41-4b51-4da8-8920-c38cf49554bf") : secret "router-metrics-certs-default" not found
Nov 04 03:17:00.669249 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:00.667996    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-mdchx" podStartSLOduration=426.667975919 podStartE2EDuration="7m6.667975919s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:09:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:17:00.667179162 +0000 UTC m=+35.496151695" watchObservedRunningTime="2026-11-04 03:17:00.667975919 +0000 UTC m=+35.496948452"
Nov 04 03:17:00.893451 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:00.893318    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-n6r7s" podStartSLOduration=426.893300114 podStartE2EDuration="7m6.893300114s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:09:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:17:00.781568038 +0000 UTC m=+35.610540572" watchObservedRunningTime="2026-11-04 03:17:00.893300114 +0000 UTC m=+35.722272646"
Nov 04 03:17:00.893630 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:00.893520    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-xpmwt" podStartSLOduration=426.893513195 podStartE2EDuration="7m6.893513195s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:09:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:17:00.892278147 +0000 UTC m=+35.721250681" watchObservedRunningTime="2026-11-04 03:17:00.893513195 +0000 UTC m=+35.722485728"
Nov 04 03:17:02.502732 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:02.501151    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1b21c41-4b51-4da8-8920-c38cf49554bf-service-ca-bundle\") pod \"router-default-7cfb798487-kbfzk\" (UID: \"c1b21c41-4b51-4da8-8920-c38cf49554bf\") " pod="openshift-ingress/router-default-7cfb798487-kbfzk"
Nov 04 03:17:02.502732 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:02.501248    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fc612506-b699-44bc-a81b-06fee4e819fe-registry-tls\") pod \"image-registry-7cb9f7c89d-wnpll\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") " pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:17:02.502732 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:02.501306    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d731646d-23ab-42d7-ae48-974929b3d2dc-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-tlhng\" (UID: \"d731646d-23ab-42d7-ae48-974929b3d2dc\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-tlhng"
Nov 04 03:17:02.502732 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:02.501346    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1b21c41-4b51-4da8-8920-c38cf49554bf-metrics-certs\") pod \"router-default-7cfb798487-kbfzk\" (UID: \"c1b21c41-4b51-4da8-8920-c38cf49554bf\") " pod="openshift-ingress/router-default-7cfb798487-kbfzk"
Nov 04 03:17:02.502732 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:02.501421    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/06698c23-887e-4ee5-9728-04098200c13d-metrics-tls\") pod \"dns-default-s4852\" (UID: \"06698c23-887e-4ee5-9728-04098200c13d\") " pod="openshift-dns/dns-default-s4852"
Nov 04 03:17:02.502732 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:02.501447    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e487dae4-8ac9-4acd-a8ad-a0d149909475-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-4z8fp\" (UID: \"e487dae4-8ac9-4acd-a8ad-a0d149909475\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-4z8fp"
Nov 04 03:17:02.502732 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:02.501481    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f1f61b42-3b07-4892-90a7-2efa024e435b-registry-tls\") pod \"image-registry-79d6dd87ff-n2jnc\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") " pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:17:02.502732 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:02.501526    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e87dded5-1e1b-45cb-ab2e-ae12550dfca7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-gmt9g\" (UID: \"e87dded5-1e1b-45cb-ab2e-ae12550dfca7\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmt9g"
Nov 04 03:17:02.502732 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:02.501669    2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 04 03:17:02.502732 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:02.501736    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e87dded5-1e1b-45cb-ab2e-ae12550dfca7-cluster-monitoring-operator-tls podName:e87dded5-1e1b-45cb-ab2e-ae12550dfca7 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:06.501716681 +0000 UTC m=+41.330689206 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e87dded5-1e1b-45cb-ab2e-ae12550dfca7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-gmt9g" (UID: "e87dded5-1e1b-45cb-ab2e-ae12550dfca7") : secret "cluster-monitoring-operator-tls" not found
Nov 04 03:17:02.502732 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:02.501816    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c1b21c41-4b51-4da8-8920-c38cf49554bf-service-ca-bundle podName:c1b21c41-4b51-4da8-8920-c38cf49554bf nodeName:}" failed. No retries permitted until 2026-11-04 03:17:06.50180258 +0000 UTC m=+41.330775095 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c1b21c41-4b51-4da8-8920-c38cf49554bf-service-ca-bundle") pod "router-default-7cfb798487-kbfzk" (UID: "c1b21c41-4b51-4da8-8920-c38cf49554bf") : configmap references non-existent config key: service-ca.crt
Nov 04 03:17:02.502732 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:02.501885    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 03:17:02.502732 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:02.501896    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cb9f7c89d-wnpll: secret "image-registry-tls" not found
Nov 04 03:17:02.502732 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:02.501927    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fc612506-b699-44bc-a81b-06fee4e819fe-registry-tls podName:fc612506-b699-44bc-a81b-06fee4e819fe nodeName:}" failed. No retries permitted until 2026-11-04 03:17:06.501916608 +0000 UTC m=+41.330889121 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fc612506-b699-44bc-a81b-06fee4e819fe-registry-tls") pod "image-registry-7cb9f7c89d-wnpll" (UID: "fc612506-b699-44bc-a81b-06fee4e819fe") : secret "image-registry-tls" not found
Nov 04 03:17:02.502732 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:02.501982    2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 04 03:17:02.504167 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:02.502012    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d731646d-23ab-42d7-ae48-974929b3d2dc-networking-console-plugin-cert podName:d731646d-23ab-42d7-ae48-974929b3d2dc nodeName:}" failed. No retries permitted until 2026-11-04 03:17:06.502003413 +0000 UTC m=+41.330975927 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d731646d-23ab-42d7-ae48-974929b3d2dc-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-tlhng" (UID: "d731646d-23ab-42d7-ae48-974929b3d2dc") : secret "networking-console-plugin-cert" not found
Nov 04 03:17:02.504167 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:02.502059    2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 04 03:17:02.504167 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:02.502090    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1b21c41-4b51-4da8-8920-c38cf49554bf-metrics-certs podName:c1b21c41-4b51-4da8-8920-c38cf49554bf nodeName:}" failed. No retries permitted until 2026-11-04 03:17:06.502080608 +0000 UTC m=+41.331053122 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c1b21c41-4b51-4da8-8920-c38cf49554bf-metrics-certs") pod "router-default-7cfb798487-kbfzk" (UID: "c1b21c41-4b51-4da8-8920-c38cf49554bf") : secret "router-metrics-certs-default" not found
Nov 04 03:17:02.504167 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:02.502140    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 03:17:02.504167 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:02.502167    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06698c23-887e-4ee5-9728-04098200c13d-metrics-tls podName:06698c23-887e-4ee5-9728-04098200c13d nodeName:}" failed. No retries permitted until 2026-11-04 03:17:06.502157598 +0000 UTC m=+41.331130114 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/06698c23-887e-4ee5-9728-04098200c13d-metrics-tls") pod "dns-default-s4852" (UID: "06698c23-887e-4ee5-9728-04098200c13d") : secret "dns-default-metrics-tls" not found
Nov 04 03:17:02.504167 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:02.502215    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 04 03:17:02.504167 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:02.502245    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e487dae4-8ac9-4acd-a8ad-a0d149909475-samples-operator-tls podName:e487dae4-8ac9-4acd-a8ad-a0d149909475 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:06.502235683 +0000 UTC m=+41.331208199 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e487dae4-8ac9-4acd-a8ad-a0d149909475-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-4z8fp" (UID: "e487dae4-8ac9-4acd-a8ad-a0d149909475") : secret "samples-operator-tls" not found
Nov 04 03:17:02.504167 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:02.502300    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 03:17:02.504167 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:02.502310    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79d6dd87ff-n2jnc: secret "image-registry-tls" not found
Nov 04 03:17:02.504167 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:02.502337    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f1f61b42-3b07-4892-90a7-2efa024e435b-registry-tls podName:f1f61b42-3b07-4892-90a7-2efa024e435b nodeName:}" failed. No retries permitted until 2026-11-04 03:17:06.502328128 +0000 UTC m=+41.331300641 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f1f61b42-3b07-4892-90a7-2efa024e435b-registry-tls") pod "image-registry-79d6dd87ff-n2jnc" (UID: "f1f61b42-3b07-4892-90a7-2efa024e435b") : secret "image-registry-tls" not found
Nov 04 03:17:02.963201 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:02.962864    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-6c2jp" event={"ID":"e451b329-bd05-44f5-9544-8283f76df9d6","Type":"ContainerStarted","Data":"769dd85e530013bcda749fa93804cb8e70832339facbd74a389e3033e14d025d"}
Nov 04 03:17:03.073596 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:03.072583    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-6c2jp" podStartSLOduration=412.724468434 podStartE2EDuration="6m55.072562514s" totalImagesPullingTime="2.34809408s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:10:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:17:03.071287192 +0000 UTC m=+37.900259747" watchObservedRunningTime="2026-11-04 03:17:03.072562514 +0000 UTC m=+37.901535049"
Nov 04 03:17:04.976616 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:04.976590    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-p9989_068619f8-7fbb-4b22-ac7d-a1f144e084c0/console-operator/0.log"
Nov 04 03:17:04.977029 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:04.976631    2579 generic.go:356] "Generic (PLEG): container finished" podID="068619f8-7fbb-4b22-ac7d-a1f144e084c0" containerID="be20b379133865d91e841bcb512f19f7c9c209994c34b8aa8662b20821a9985c" exitCode=255
Nov 04 03:17:04.977029 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:04.976753    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-p9989" event={"ID":"068619f8-7fbb-4b22-ac7d-a1f144e084c0","Type":"ContainerDied","Data":"be20b379133865d91e841bcb512f19f7c9c209994c34b8aa8662b20821a9985c"}
Nov 04 03:17:04.977124 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:04.977113    2579 scope.go:122] "RemoveContainer" containerID="be20b379133865d91e841bcb512f19f7c9c209994c34b8aa8662b20821a9985c"
Nov 04 03:17:04.982285 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:04.982246    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dln2k" event={"ID":"6dedccdc-aea2-4664-8480-ca3c9359437d","Type":"ContainerStarted","Data":"a76b21e698659465e2efb560a01e2d433a6be7674c32fed65285c373cf0f5d0a"}
Nov 04 03:17:04.984791 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:04.984753    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-hsb49" event={"ID":"3b178ce9-e140-4420-81f1-627de00b90c6","Type":"ContainerStarted","Data":"79b17768c249ad4244c1d5a0fb2f9a7933644b30ade14ed06d2c721b7c397fe9"}
Nov 04 03:17:05.218996 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:05.217827    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-operator-7c4986d449-hsb49" podStartSLOduration=406.164523384 podStartE2EDuration="6m51.217813292s" totalImagesPullingTime="5.053289908s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:10:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:17:05.21724793 +0000 UTC m=+40.046220462" watchObservedRunningTime="2026-11-04 03:17:05.217813292 +0000 UTC m=+40.046785842"
Nov 04 03:17:05.328144 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:05.328091    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dln2k" podStartSLOduration=413.186213257 podStartE2EDuration="6m58.328067867s" totalImagesPullingTime="5.14185461s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:10:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:17:05.327595018 +0000 UTC m=+40.156567541" watchObservedRunningTime="2026-11-04 03:17:05.328067867 +0000 UTC m=+40.157040402"
Nov 04 03:17:05.746523 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:05.746490    2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-46btb"]
Nov 04 03:17:05.765774 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:05.765742    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-46btb"
Nov 04 03:17:05.825123 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:05.825088    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-46btb"]
Nov 04 03:17:05.838393 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:05.838303    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fhxt\" (UniqueName: \"kubernetes.io/projected/536a0057-a438-4ad9-a4cd-b26cb7f3c645-kube-api-access-7fhxt\") pod \"insights-runtime-extractor-46btb\" (UID: \"536a0057-a438-4ad9-a4cd-b26cb7f3c645\") " pod="openshift-insights/insights-runtime-extractor-46btb"
Nov 04 03:17:05.838393 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:05.838364    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/536a0057-a438-4ad9-a4cd-b26cb7f3c645-crio-socket\") pod \"insights-runtime-extractor-46btb\" (UID: \"536a0057-a438-4ad9-a4cd-b26cb7f3c645\") " pod="openshift-insights/insights-runtime-extractor-46btb"
Nov 04 03:17:05.838665 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:05.838598    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/536a0057-a438-4ad9-a4cd-b26cb7f3c645-data-volume\") pod \"insights-runtime-extractor-46btb\" (UID: \"536a0057-a438-4ad9-a4cd-b26cb7f3c645\") " pod="openshift-insights/insights-runtime-extractor-46btb"
Nov 04 03:17:05.838735 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:05.838673    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/536a0057-a438-4ad9-a4cd-b26cb7f3c645-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-46btb\" (UID: \"536a0057-a438-4ad9-a4cd-b26cb7f3c645\") " pod="openshift-insights/insights-runtime-extractor-46btb"
Nov 04 03:17:05.838788 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:05.838770    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/536a0057-a438-4ad9-a4cd-b26cb7f3c645-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-46btb\" (UID: \"536a0057-a438-4ad9-a4cd-b26cb7f3c645\") " pod="openshift-insights/insights-runtime-extractor-46btb"
Nov 04 03:17:05.880196 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:05.880141    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 04 03:17:05.880196 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:05.880141    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 04 03:17:05.880459 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:05.880297    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jw2mq\""
Nov 04 03:17:05.940239 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:05.940203    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fhxt\" (UniqueName: \"kubernetes.io/projected/536a0057-a438-4ad9-a4cd-b26cb7f3c645-kube-api-access-7fhxt\") pod \"insights-runtime-extractor-46btb\" (UID: \"536a0057-a438-4ad9-a4cd-b26cb7f3c645\") " pod="openshift-insights/insights-runtime-extractor-46btb"
Nov 04 03:17:05.940459 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:05.940271    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/536a0057-a438-4ad9-a4cd-b26cb7f3c645-crio-socket\") pod \"insights-runtime-extractor-46btb\" (UID: \"536a0057-a438-4ad9-a4cd-b26cb7f3c645\") " pod="openshift-insights/insights-runtime-extractor-46btb"
Nov 04 03:17:05.940459 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:05.940361    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/536a0057-a438-4ad9-a4cd-b26cb7f3c645-data-volume\") pod \"insights-runtime-extractor-46btb\" (UID: \"536a0057-a438-4ad9-a4cd-b26cb7f3c645\") " pod="openshift-insights/insights-runtime-extractor-46btb"
Nov 04 03:17:05.940641 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:05.940617    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/536a0057-a438-4ad9-a4cd-b26cb7f3c645-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-46btb\" (UID: \"536a0057-a438-4ad9-a4cd-b26cb7f3c645\") " pod="openshift-insights/insights-runtime-extractor-46btb"
Nov 04 03:17:05.940785 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:05.940703    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/536a0057-a438-4ad9-a4cd-b26cb7f3c645-crio-socket\") pod \"insights-runtime-extractor-46btb\" (UID: \"536a0057-a438-4ad9-a4cd-b26cb7f3c645\") " pod="openshift-insights/insights-runtime-extractor-46btb"
Nov 04 03:17:05.940785 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:05.940709    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/536a0057-a438-4ad9-a4cd-b26cb7f3c645-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-46btb\" (UID: \"536a0057-a438-4ad9-a4cd-b26cb7f3c645\") " pod="openshift-insights/insights-runtime-extractor-46btb"
Nov 04 03:17:05.940886 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:05.940799    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:05.940886 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:05.940848    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/536a0057-a438-4ad9-a4cd-b26cb7f3c645-insights-runtime-extractor-tls podName:536a0057-a438-4ad9-a4cd-b26cb7f3c645 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:06.440831163 +0000 UTC m=+41.269803675 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/536a0057-a438-4ad9-a4cd-b26cb7f3c645-insights-runtime-extractor-tls") pod "insights-runtime-extractor-46btb" (UID: "536a0057-a438-4ad9-a4cd-b26cb7f3c645") : secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:05.940987 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:05.940952    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 04 03:17:05.941039 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:05.940995    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/536a0057-a438-4ad9-a4cd-b26cb7f3c645-kube-rbac-proxy-cm podName:536a0057-a438-4ad9-a4cd-b26cb7f3c645 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:06.440981561 +0000 UTC m=+41.269954086 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/536a0057-a438-4ad9-a4cd-b26cb7f3c645-kube-rbac-proxy-cm") pod "insights-runtime-extractor-46btb" (UID: "536a0057-a438-4ad9-a4cd-b26cb7f3c645") : configmap "kube-rbac-proxy" not found
Nov 04 03:17:05.941039 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:05.941019    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/536a0057-a438-4ad9-a4cd-b26cb7f3c645-data-volume\") pod \"insights-runtime-extractor-46btb\" (UID: \"536a0057-a438-4ad9-a4cd-b26cb7f3c645\") " pod="openshift-insights/insights-runtime-extractor-46btb"
Nov 04 03:17:05.991388 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:05.991345    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qbskg" event={"ID":"55c4af4c-188b-495b-9052-5700a1f60602","Type":"ContainerStarted","Data":"98db46e7294e32beed878188e39548af189b79999615060ee5e4b95ae5d1012b"}
Nov 04 03:17:05.993377 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:05.993346    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-p9989_068619f8-7fbb-4b22-ac7d-a1f144e084c0/console-operator/1.log"
Nov 04 03:17:05.993904 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:05.993887    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-p9989_068619f8-7fbb-4b22-ac7d-a1f144e084c0/console-operator/0.log"
Nov 04 03:17:05.994018 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:05.993931    2579 generic.go:356] "Generic (PLEG): container finished" podID="068619f8-7fbb-4b22-ac7d-a1f144e084c0" containerID="9075272d4964f13dc75390adcbd26ac4ef501c2d09e16c73075206e396a8a4b3" exitCode=255
Nov 04 03:17:05.994084 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:05.994023    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-p9989" event={"ID":"068619f8-7fbb-4b22-ac7d-a1f144e084c0","Type":"ContainerDied","Data":"9075272d4964f13dc75390adcbd26ac4ef501c2d09e16c73075206e396a8a4b3"}
Nov 04 03:17:05.994084 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:05.994079    2579 scope.go:122] "RemoveContainer" containerID="be20b379133865d91e841bcb512f19f7c9c209994c34b8aa8662b20821a9985c"
Nov 04 03:17:05.994681 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:05.994365    2579 scope.go:122] "RemoveContainer" containerID="9075272d4964f13dc75390adcbd26ac4ef501c2d09e16c73075206e396a8a4b3"
Nov 04 03:17:05.994681 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:05.994634    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-p9989_openshift-console-operator(068619f8-7fbb-4b22-ac7d-a1f144e084c0)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-p9989" podUID="068619f8-7fbb-4b22-ac7d-a1f144e084c0"
Nov 04 03:17:05.996222 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:05.996181    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-js5xd" event={"ID":"0beb091e-5c2f-4136-93a2-771f9b861411","Type":"ContainerStarted","Data":"a6c981ad2470dd6f32e35ca04b810224a433e5915ee5d4d759e922e026a06479"}
Nov 04 03:17:06.006878 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:06.006647    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fhxt\" (UniqueName: \"kubernetes.io/projected/536a0057-a438-4ad9-a4cd-b26cb7f3c645-kube-api-access-7fhxt\") pod \"insights-runtime-extractor-46btb\" (UID: \"536a0057-a438-4ad9-a4cd-b26cb7f3c645\") " pod="openshift-insights/insights-runtime-extractor-46btb"
Nov 04 03:17:06.110179 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:06.110118    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-qbskg" podStartSLOduration=2.246654618 podStartE2EDuration="8.11009782s" totalImagesPullingTime="5.863443202s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:16:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:17:06.107477797 +0000 UTC m=+40.936450333" watchObservedRunningTime="2026-11-04 03:17:06.11009782 +0000 UTC m=+40.939070354"
Nov 04 03:17:06.327617 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:06.327566    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-js5xd" podStartSLOduration=406.035994719 podStartE2EDuration="6m51.327551613s" totalImagesPullingTime="5.291556894s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:10:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:17:06.21631482 +0000 UTC m=+41.045287352" watchObservedRunningTime="2026-11-04 03:17:06.327551613 +0000 UTC m=+41.156524163"
Nov 04 03:17:06.448284 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:06.448243    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/536a0057-a438-4ad9-a4cd-b26cb7f3c645-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-46btb\" (UID: \"536a0057-a438-4ad9-a4cd-b26cb7f3c645\") " pod="openshift-insights/insights-runtime-extractor-46btb"
Nov 04 03:17:06.448470 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:06.448350    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/536a0057-a438-4ad9-a4cd-b26cb7f3c645-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-46btb\" (UID: \"536a0057-a438-4ad9-a4cd-b26cb7f3c645\") " pod="openshift-insights/insights-runtime-extractor-46btb"
Nov 04 03:17:06.448470 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:06.448449    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:06.448550 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:06.448484    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 04 03:17:06.448550 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:06.448516    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/536a0057-a438-4ad9-a4cd-b26cb7f3c645-insights-runtime-extractor-tls podName:536a0057-a438-4ad9-a4cd-b26cb7f3c645 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:07.448501574 +0000 UTC m=+42.277474090 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/536a0057-a438-4ad9-a4cd-b26cb7f3c645-insights-runtime-extractor-tls") pod "insights-runtime-extractor-46btb" (UID: "536a0057-a438-4ad9-a4cd-b26cb7f3c645") : secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:06.448550 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:06.448539    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/536a0057-a438-4ad9-a4cd-b26cb7f3c645-kube-rbac-proxy-cm podName:536a0057-a438-4ad9-a4cd-b26cb7f3c645 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:07.448525601 +0000 UTC m=+42.277498140 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/536a0057-a438-4ad9-a4cd-b26cb7f3c645-kube-rbac-proxy-cm") pod "insights-runtime-extractor-46btb" (UID: "536a0057-a438-4ad9-a4cd-b26cb7f3c645") : configmap "kube-rbac-proxy" not found
Nov 04 03:17:06.548976 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:06.548883    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e87dded5-1e1b-45cb-ab2e-ae12550dfca7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-gmt9g\" (UID: \"e87dded5-1e1b-45cb-ab2e-ae12550dfca7\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmt9g"
Nov 04 03:17:06.548976 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:06.548938    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1b21c41-4b51-4da8-8920-c38cf49554bf-service-ca-bundle\") pod \"router-default-7cfb798487-kbfzk\" (UID: \"c1b21c41-4b51-4da8-8920-c38cf49554bf\") " pod="openshift-ingress/router-default-7cfb798487-kbfzk"
Nov 04 03:17:06.549214 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:06.549041    2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 04 03:17:06.549214 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:06.549057    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c1b21c41-4b51-4da8-8920-c38cf49554bf-service-ca-bundle podName:c1b21c41-4b51-4da8-8920-c38cf49554bf nodeName:}" failed. No retries permitted until 2026-11-04 03:17:14.549038996 +0000 UTC m=+49.378011521 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c1b21c41-4b51-4da8-8920-c38cf49554bf-service-ca-bundle") pod "router-default-7cfb798487-kbfzk" (UID: "c1b21c41-4b51-4da8-8920-c38cf49554bf") : configmap references non-existent config key: service-ca.crt
Nov 04 03:17:06.549214 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:06.549078    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fc612506-b699-44bc-a81b-06fee4e819fe-registry-tls\") pod \"image-registry-7cb9f7c89d-wnpll\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") " pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:17:06.549214 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:06.549089    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e87dded5-1e1b-45cb-ab2e-ae12550dfca7-cluster-monitoring-operator-tls podName:e87dded5-1e1b-45cb-ab2e-ae12550dfca7 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:14.54907762 +0000 UTC m=+49.378050151 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e87dded5-1e1b-45cb-ab2e-ae12550dfca7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-gmt9g" (UID: "e87dded5-1e1b-45cb-ab2e-ae12550dfca7") : secret "cluster-monitoring-operator-tls" not found
Nov 04 03:17:06.549214 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:06.549139    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d731646d-23ab-42d7-ae48-974929b3d2dc-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-tlhng\" (UID: \"d731646d-23ab-42d7-ae48-974929b3d2dc\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-tlhng"
Nov 04 03:17:06.549214 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:06.549169    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1b21c41-4b51-4da8-8920-c38cf49554bf-metrics-certs\") pod \"router-default-7cfb798487-kbfzk\" (UID: \"c1b21c41-4b51-4da8-8920-c38cf49554bf\") " pod="openshift-ingress/router-default-7cfb798487-kbfzk"
Nov 04 03:17:06.549214 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:06.549144    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 03:17:06.549214 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:06.549216    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/06698c23-887e-4ee5-9728-04098200c13d-metrics-tls\") pod \"dns-default-s4852\" (UID: \"06698c23-887e-4ee5-9728-04098200c13d\") " pod="openshift-dns/dns-default-s4852"
Nov 04 03:17:06.549605 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:06.549220    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cb9f7c89d-wnpll: secret "image-registry-tls" not found
Nov 04 03:17:06.549605 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:06.549238    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e487dae4-8ac9-4acd-a8ad-a0d149909475-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-4z8fp\" (UID: \"e487dae4-8ac9-4acd-a8ad-a0d149909475\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-4z8fp"
Nov 04 03:17:06.549605 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:06.549274    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fc612506-b699-44bc-a81b-06fee4e819fe-registry-tls podName:fc612506-b699-44bc-a81b-06fee4e819fe nodeName:}" failed. No retries permitted until 2026-11-04 03:17:14.549257711 +0000 UTC m=+49.378230225 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fc612506-b699-44bc-a81b-06fee4e819fe-registry-tls") pod "image-registry-7cb9f7c89d-wnpll" (UID: "fc612506-b699-44bc-a81b-06fee4e819fe") : secret "image-registry-tls" not found
Nov 04 03:17:06.549605 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:06.549196    2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 04 03:17:06.549605 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:06.549310    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 04 03:17:06.549605 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:06.549343    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e487dae4-8ac9-4acd-a8ad-a0d149909475-samples-operator-tls podName:e487dae4-8ac9-4acd-a8ad-a0d149909475 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:14.549332271 +0000 UTC m=+49.378304796 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e487dae4-8ac9-4acd-a8ad-a0d149909475-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-4z8fp" (UID: "e487dae4-8ac9-4acd-a8ad-a0d149909475") : secret "samples-operator-tls" not found
Nov 04 03:17:06.549605 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:06.549344    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 03:17:06.549605 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:06.549351    2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 04 03:17:06.549605 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:06.549367    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d731646d-23ab-42d7-ae48-974929b3d2dc-networking-console-plugin-cert podName:d731646d-23ab-42d7-ae48-974929b3d2dc nodeName:}" failed. No retries permitted until 2026-11-04 03:17:14.549350844 +0000 UTC m=+49.378323360 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d731646d-23ab-42d7-ae48-974929b3d2dc-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-tlhng" (UID: "d731646d-23ab-42d7-ae48-974929b3d2dc") : secret "networking-console-plugin-cert" not found
Nov 04 03:17:06.549605 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:06.549387    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06698c23-887e-4ee5-9728-04098200c13d-metrics-tls podName:06698c23-887e-4ee5-9728-04098200c13d nodeName:}" failed. No retries permitted until 2026-11-04 03:17:14.549377634 +0000 UTC m=+49.378350143 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/06698c23-887e-4ee5-9728-04098200c13d-metrics-tls") pod "dns-default-s4852" (UID: "06698c23-887e-4ee5-9728-04098200c13d") : secret "dns-default-metrics-tls" not found
Nov 04 03:17:06.549605 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:06.549298    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f1f61b42-3b07-4892-90a7-2efa024e435b-registry-tls\") pod \"image-registry-79d6dd87ff-n2jnc\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") " pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:17:06.549605 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:06.549367    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 03:17:06.549605 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:06.549424    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1b21c41-4b51-4da8-8920-c38cf49554bf-metrics-certs podName:c1b21c41-4b51-4da8-8920-c38cf49554bf nodeName:}" failed. No retries permitted until 2026-11-04 03:17:14.54939394 +0000 UTC m=+49.378366457 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c1b21c41-4b51-4da8-8920-c38cf49554bf-metrics-certs") pod "router-default-7cfb798487-kbfzk" (UID: "c1b21c41-4b51-4da8-8920-c38cf49554bf") : secret "router-metrics-certs-default" not found
Nov 04 03:17:06.549605 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:06.549432    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79d6dd87ff-n2jnc: secret "image-registry-tls" not found
Nov 04 03:17:06.549605 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:06.549491    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f1f61b42-3b07-4892-90a7-2efa024e435b-registry-tls podName:f1f61b42-3b07-4892-90a7-2efa024e435b nodeName:}" failed. No retries permitted until 2026-11-04 03:17:14.549476945 +0000 UTC m=+49.378449469 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f1f61b42-3b07-4892-90a7-2efa024e435b-registry-tls") pod "image-registry-79d6dd87ff-n2jnc" (UID: "f1f61b42-3b07-4892-90a7-2efa024e435b") : secret "image-registry-tls" not found
Nov 04 03:17:07.000729 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:07.000694    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-7lkrx" event={"ID":"cc469246-2293-4301-9e46-bf9d609be964","Type":"ContainerStarted","Data":"4aa7ff99da75c342130ff894bdfec1b5d208296ef28179076dda53454dc9e047"}
Nov 04 03:17:07.002433 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:07.002389    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-p9989_068619f8-7fbb-4b22-ac7d-a1f144e084c0/console-operator/1.log"
Nov 04 03:17:07.002751 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:07.002718    2579 scope.go:122] "RemoveContainer" containerID="9075272d4964f13dc75390adcbd26ac4ef501c2d09e16c73075206e396a8a4b3"
Nov 04 03:17:07.002928 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:07.002909    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-p9989_openshift-console-operator(068619f8-7fbb-4b22-ac7d-a1f144e084c0)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-p9989" podUID="068619f8-7fbb-4b22-ac7d-a1f144e084c0"
Nov 04 03:17:07.004106 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:07.004081    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hjssc" event={"ID":"04b955ef-528b-4573-b18f-0957658f79e4","Type":"ContainerStarted","Data":"e506cde98da17fac21d04ec9838917734221e83d95bf977942f76a63ddb6e547"}
Nov 04 03:17:07.004216 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:07.004178    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-hjssc"
Nov 04 03:17:07.110316 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:07.110243    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-7lkrx" podStartSLOduration=378.500415405 podStartE2EDuration="6m25.110222852s" totalImagesPullingTime="6.609807447s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:10:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:17:07.109257486 +0000 UTC m=+41.938230018" watchObservedRunningTime="2026-11-04 03:17:07.110222852 +0000 UTC m=+41.939195388"
Nov 04 03:17:07.220100 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:07.219871    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-hjssc" podStartSLOduration=34.442670894 podStartE2EDuration="41.219856255s" totalImagesPullingTime="6.777185361s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:16:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:17:07.218993608 +0000 UTC m=+42.047966141" watchObservedRunningTime="2026-11-04 03:17:07.219856255 +0000 UTC m=+42.048828788"
Nov 04 03:17:07.460237 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:07.460200    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/536a0057-a438-4ad9-a4cd-b26cb7f3c645-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-46btb\" (UID: \"536a0057-a438-4ad9-a4cd-b26cb7f3c645\") " pod="openshift-insights/insights-runtime-extractor-46btb"
Nov 04 03:17:07.460515 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:07.460329    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/536a0057-a438-4ad9-a4cd-b26cb7f3c645-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-46btb\" (UID: \"536a0057-a438-4ad9-a4cd-b26cb7f3c645\") " pod="openshift-insights/insights-runtime-extractor-46btb"
Nov 04 03:17:07.460515 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:07.460349    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 04 03:17:07.460515 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:07.460437    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:07.460515 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:07.460453    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/536a0057-a438-4ad9-a4cd-b26cb7f3c645-kube-rbac-proxy-cm podName:536a0057-a438-4ad9-a4cd-b26cb7f3c645 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:09.460430223 +0000 UTC m=+44.289402734 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/536a0057-a438-4ad9-a4cd-b26cb7f3c645-kube-rbac-proxy-cm") pod "insights-runtime-extractor-46btb" (UID: "536a0057-a438-4ad9-a4cd-b26cb7f3c645") : configmap "kube-rbac-proxy" not found
Nov 04 03:17:07.460515 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:07.460475    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/536a0057-a438-4ad9-a4cd-b26cb7f3c645-insights-runtime-extractor-tls podName:536a0057-a438-4ad9-a4cd-b26cb7f3c645 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:09.460464753 +0000 UTC m=+44.289437284 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/536a0057-a438-4ad9-a4cd-b26cb7f3c645-insights-runtime-extractor-tls") pod "insights-runtime-extractor-46btb" (UID: "536a0057-a438-4ad9-a4cd-b26cb7f3c645") : secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:09.051845 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:09.051808    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-p9989"
Nov 04 03:17:09.052272 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:09.051887    2579 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-p9989"
Nov 04 03:17:09.052272 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:09.052218    2579 scope.go:122] "RemoveContainer" containerID="9075272d4964f13dc75390adcbd26ac4ef501c2d09e16c73075206e396a8a4b3"
Nov 04 03:17:09.052437 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:09.052394    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-p9989_openshift-console-operator(068619f8-7fbb-4b22-ac7d-a1f144e084c0)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-p9989" podUID="068619f8-7fbb-4b22-ac7d-a1f144e084c0"
Nov 04 03:17:09.479147 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:09.479104    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/536a0057-a438-4ad9-a4cd-b26cb7f3c645-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-46btb\" (UID: \"536a0057-a438-4ad9-a4cd-b26cb7f3c645\") " pod="openshift-insights/insights-runtime-extractor-46btb"
Nov 04 03:17:09.479319 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:09.479181    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/536a0057-a438-4ad9-a4cd-b26cb7f3c645-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-46btb\" (UID: \"536a0057-a438-4ad9-a4cd-b26cb7f3c645\") " pod="openshift-insights/insights-runtime-extractor-46btb"
Nov 04 03:17:09.479319 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:09.479256    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 04 03:17:09.479319 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:09.479257    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:09.479319 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:09.479308    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/536a0057-a438-4ad9-a4cd-b26cb7f3c645-kube-rbac-proxy-cm podName:536a0057-a438-4ad9-a4cd-b26cb7f3c645 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:13.479295522 +0000 UTC m=+48.308268045 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/536a0057-a438-4ad9-a4cd-b26cb7f3c645-kube-rbac-proxy-cm") pod "insights-runtime-extractor-46btb" (UID: "536a0057-a438-4ad9-a4cd-b26cb7f3c645") : configmap "kube-rbac-proxy" not found
Nov 04 03:17:09.479319 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:09.479321    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/536a0057-a438-4ad9-a4cd-b26cb7f3c645-insights-runtime-extractor-tls podName:536a0057-a438-4ad9-a4cd-b26cb7f3c645 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:13.479315189 +0000 UTC m=+48.308287699 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/536a0057-a438-4ad9-a4cd-b26cb7f3c645-insights-runtime-extractor-tls") pod "insights-runtime-extractor-46btb" (UID: "536a0057-a438-4ad9-a4cd-b26cb7f3c645") : secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:13.340072 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:13.340033    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6pwhj_aab0bd9b-cb90-4d9b-9f46-52c6c87a34cf/dns-node-resolver/0.log"
Nov 04 03:17:13.514945 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:13.514904    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/536a0057-a438-4ad9-a4cd-b26cb7f3c645-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-46btb\" (UID: \"536a0057-a438-4ad9-a4cd-b26cb7f3c645\") " pod="openshift-insights/insights-runtime-extractor-46btb"
Nov 04 03:17:13.515122 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:13.514982    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/536a0057-a438-4ad9-a4cd-b26cb7f3c645-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-46btb\" (UID: \"536a0057-a438-4ad9-a4cd-b26cb7f3c645\") " pod="openshift-insights/insights-runtime-extractor-46btb"
Nov 04 03:17:13.515122 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:13.515054    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 04 03:17:13.515122 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:13.515081    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:13.515122 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:13.515110    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/536a0057-a438-4ad9-a4cd-b26cb7f3c645-kube-rbac-proxy-cm podName:536a0057-a438-4ad9-a4cd-b26cb7f3c645 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:21.515095041 +0000 UTC m=+56.344067564 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/536a0057-a438-4ad9-a4cd-b26cb7f3c645-kube-rbac-proxy-cm") pod "insights-runtime-extractor-46btb" (UID: "536a0057-a438-4ad9-a4cd-b26cb7f3c645") : configmap "kube-rbac-proxy" not found
Nov 04 03:17:13.515307 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:13.515160    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/536a0057-a438-4ad9-a4cd-b26cb7f3c645-insights-runtime-extractor-tls podName:536a0057-a438-4ad9-a4cd-b26cb7f3c645 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:21.51514205 +0000 UTC m=+56.344114572 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/536a0057-a438-4ad9-a4cd-b26cb7f3c645-insights-runtime-extractor-tls") pod "insights-runtime-extractor-46btb" (UID: "536a0057-a438-4ad9-a4cd-b26cb7f3c645") : secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:14.540338 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:14.540309    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-n4b5s_7713890e-763d-4a51-bffb-6cf8dd12fd18/node-ca/0.log"
Nov 04 03:17:14.625754 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:14.625716    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fc612506-b699-44bc-a81b-06fee4e819fe-registry-tls\") pod \"image-registry-7cb9f7c89d-wnpll\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") " pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:17:14.625913 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:14.625769    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d731646d-23ab-42d7-ae48-974929b3d2dc-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-tlhng\" (UID: \"d731646d-23ab-42d7-ae48-974929b3d2dc\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-tlhng"
Nov 04 03:17:14.625913 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:14.625802    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1b21c41-4b51-4da8-8920-c38cf49554bf-metrics-certs\") pod \"router-default-7cfb798487-kbfzk\" (UID: \"c1b21c41-4b51-4da8-8920-c38cf49554bf\") " pod="openshift-ingress/router-default-7cfb798487-kbfzk"
Nov 04 03:17:14.625913 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:14.625841    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/06698c23-887e-4ee5-9728-04098200c13d-metrics-tls\") pod \"dns-default-s4852\" (UID: \"06698c23-887e-4ee5-9728-04098200c13d\") " pod="openshift-dns/dns-default-s4852"
Nov 04 03:17:14.625913 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:14.625858    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e487dae4-8ac9-4acd-a8ad-a0d149909475-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-4z8fp\" (UID: \"e487dae4-8ac9-4acd-a8ad-a0d149909475\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-4z8fp"
Nov 04 03:17:14.625913 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:14.625890    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f1f61b42-3b07-4892-90a7-2efa024e435b-registry-tls\") pod \"image-registry-79d6dd87ff-n2jnc\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") " pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:17:14.626190 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:14.625936    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e87dded5-1e1b-45cb-ab2e-ae12550dfca7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-gmt9g\" (UID: \"e87dded5-1e1b-45cb-ab2e-ae12550dfca7\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmt9g"
Nov 04 03:17:14.626190 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:14.625975    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1b21c41-4b51-4da8-8920-c38cf49554bf-service-ca-bundle\") pod \"router-default-7cfb798487-kbfzk\" (UID: \"c1b21c41-4b51-4da8-8920-c38cf49554bf\") " pod="openshift-ingress/router-default-7cfb798487-kbfzk"
Nov 04 03:17:14.626190 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:14.625986    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 04 03:17:14.626190 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:14.625986    2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 04 03:17:14.626190 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:14.626116    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e487dae4-8ac9-4acd-a8ad-a0d149909475-samples-operator-tls podName:e487dae4-8ac9-4acd-a8ad-a0d149909475 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:30.626091432 +0000 UTC m=+65.455063960 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e487dae4-8ac9-4acd-a8ad-a0d149909475-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-4z8fp" (UID: "e487dae4-8ac9-4acd-a8ad-a0d149909475") : secret "samples-operator-tls" not found
Nov 04 03:17:14.626190 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:14.626135    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c1b21c41-4b51-4da8-8920-c38cf49554bf-service-ca-bundle podName:c1b21c41-4b51-4da8-8920-c38cf49554bf nodeName:}" failed. No retries permitted until 2026-11-04 03:17:30.626125536 +0000 UTC m=+65.455098069 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c1b21c41-4b51-4da8-8920-c38cf49554bf-service-ca-bundle") pod "router-default-7cfb798487-kbfzk" (UID: "c1b21c41-4b51-4da8-8920-c38cf49554bf") : configmap references non-existent config key: service-ca.crt
Nov 04 03:17:14.626190 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:14.626144    2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 04 03:17:14.626190 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:14.626160    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1b21c41-4b51-4da8-8920-c38cf49554bf-metrics-certs podName:c1b21c41-4b51-4da8-8920-c38cf49554bf nodeName:}" failed. No retries permitted until 2026-11-04 03:17:30.626141435 +0000 UTC m=+65.455113945 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c1b21c41-4b51-4da8-8920-c38cf49554bf-metrics-certs") pod "router-default-7cfb798487-kbfzk" (UID: "c1b21c41-4b51-4da8-8920-c38cf49554bf") : secret "router-metrics-certs-default" not found
Nov 04 03:17:14.626190 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:14.626188    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e87dded5-1e1b-45cb-ab2e-ae12550dfca7-cluster-monitoring-operator-tls podName:e87dded5-1e1b-45cb-ab2e-ae12550dfca7 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:30.626173396 +0000 UTC m=+65.455145921 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e87dded5-1e1b-45cb-ab2e-ae12550dfca7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-gmt9g" (UID: "e87dded5-1e1b-45cb-ab2e-ae12550dfca7") : secret "cluster-monitoring-operator-tls" not found
Nov 04 03:17:14.629029 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:14.629004    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/06698c23-887e-4ee5-9728-04098200c13d-metrics-tls\") pod \"dns-default-s4852\" (UID: \"06698c23-887e-4ee5-9728-04098200c13d\") " pod="openshift-dns/dns-default-s4852"
Nov 04 03:17:14.629139 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:14.629054    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fc612506-b699-44bc-a81b-06fee4e819fe-registry-tls\") pod \"image-registry-7cb9f7c89d-wnpll\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") " pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:17:14.629221 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:14.629202    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f1f61b42-3b07-4892-90a7-2efa024e435b-registry-tls\") pod \"image-registry-79d6dd87ff-n2jnc\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") " pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:17:14.629331 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:14.629310    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d731646d-23ab-42d7-ae48-974929b3d2dc-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-tlhng\" (UID: \"d731646d-23ab-42d7-ae48-974929b3d2dc\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-tlhng"
Nov 04 03:17:14.642273 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:14.642248    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:17:14.671752 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:14.671721    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-tlhng"
Nov 04 03:17:14.696761 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:14.696741    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-s4852"
Nov 04 03:17:14.897421 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:14.897366    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:17:15.161690 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:15.161607    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-tlhng"]
Nov 04 03:17:15.164802 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:15.164774    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-s4852"]
Nov 04 03:17:15.169242 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:17:15.169144    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd731646d_23ab_42d7_ae48_974929b3d2dc.slice/crio-01166c007db55d3b820092642ea9c65c883b12c05090bf951978d4a1b1531bd8 WatchSource:0}: Error finding container 01166c007db55d3b820092642ea9c65c883b12c05090bf951978d4a1b1531bd8: Status 404 returned error can't find the container with id 01166c007db55d3b820092642ea9c65c883b12c05090bf951978d4a1b1531bd8
Nov 04 03:17:15.171390 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:17:15.171372    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06698c23_887e_4ee5_9728_04098200c13d.slice/crio-c6b9d14b9e7cc3372ca72b16275cf82ba5230ebee08be64970e3b3bc50af00e8 WatchSource:0}: Error finding container c6b9d14b9e7cc3372ca72b16275cf82ba5230ebee08be64970e3b3bc50af00e8: Status 404 returned error can't find the container with id c6b9d14b9e7cc3372ca72b16275cf82ba5230ebee08be64970e3b3bc50af00e8
Nov 04 03:17:15.208288 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:15.208263    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-79d6dd87ff-n2jnc"]
Nov 04 03:17:15.209596 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:17:15.209568    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1f61b42_3b07_4892_90a7_2efa024e435b.slice/crio-162b5150b1fa0c5491cabb76b570878f66758b46a19ba0e60cd6b5de9f2a4708 WatchSource:0}: Error finding container 162b5150b1fa0c5491cabb76b570878f66758b46a19ba0e60cd6b5de9f2a4708: Status 404 returned error can't find the container with id 162b5150b1fa0c5491cabb76b570878f66758b46a19ba0e60cd6b5de9f2a4708
Nov 04 03:17:15.294476 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:15.294442    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7cb9f7c89d-wnpll"]
Nov 04 03:17:15.301563 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:17:15.301537    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc612506_b699_44bc_a81b_06fee4e819fe.slice/crio-b0d8f5dcca3cd28fec67127e455a70a4dd57c277871218844baaa61bc2e7b915 WatchSource:0}: Error finding container b0d8f5dcca3cd28fec67127e455a70a4dd57c277871218844baaa61bc2e7b915: Status 404 returned error can't find the container with id b0d8f5dcca3cd28fec67127e455a70a4dd57c277871218844baaa61bc2e7b915
Nov 04 03:17:15.339885 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:15.339851    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qbskg_55c4af4c-188b-495b-9052-5700a1f60602/serve-healthcheck-canary/0.log"
Nov 04 03:17:16.032631 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:16.032588    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-s4852" event={"ID":"06698c23-887e-4ee5-9728-04098200c13d","Type":"ContainerStarted","Data":"c6b9d14b9e7cc3372ca72b16275cf82ba5230ebee08be64970e3b3bc50af00e8"}
Nov 04 03:17:16.033834 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:16.033804    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-tlhng" event={"ID":"d731646d-23ab-42d7-ae48-974929b3d2dc","Type":"ContainerStarted","Data":"01166c007db55d3b820092642ea9c65c883b12c05090bf951978d4a1b1531bd8"}
Nov 04 03:17:16.035169 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:16.035145    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc" event={"ID":"f1f61b42-3b07-4892-90a7-2efa024e435b","Type":"ContainerStarted","Data":"5140cf4d30301289198ac7d930beec469d79386b2c6405fb1a5c7ebcba51038d"}
Nov 04 03:17:16.035284 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:16.035190    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc" event={"ID":"f1f61b42-3b07-4892-90a7-2efa024e435b","Type":"ContainerStarted","Data":"162b5150b1fa0c5491cabb76b570878f66758b46a19ba0e60cd6b5de9f2a4708"}
Nov 04 03:17:16.035284 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:16.035252    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:17:16.036786 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:16.036763    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll" event={"ID":"fc612506-b699-44bc-a81b-06fee4e819fe","Type":"ContainerStarted","Data":"5896ef33504444ee786b2836a14d4aaf1f015c80c7ed4a72e6918338be5342a0"}
Nov 04 03:17:16.036889 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:16.036801    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll" event={"ID":"fc612506-b699-44bc-a81b-06fee4e819fe","Type":"ContainerStarted","Data":"b0d8f5dcca3cd28fec67127e455a70a4dd57c277871218844baaa61bc2e7b915"}
Nov 04 03:17:16.036933 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:16.036912    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:17:16.141095 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:16.141059    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7d4bc45dd6-dln2k_6dedccdc-aea2-4664-8480-ca3c9359437d/kube-storage-version-migrator-operator/0.log"
Nov 04 03:17:16.150510 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:16.150455    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc" podStartSLOduration=46.15044255 podStartE2EDuration="46.15044255s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:16:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:17:16.149290203 +0000 UTC m=+50.978262745" watchObservedRunningTime="2026-11-04 03:17:16.15044255 +0000 UTC m=+50.979415076"
Nov 04 03:17:16.264068 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:16.264011    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll" podStartSLOduration=420.26399581 podStartE2EDuration="7m0.26399581s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:10:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:17:16.263110175 +0000 UTC m=+51.092082707" watchObservedRunningTime="2026-11-04 03:17:16.26399581 +0000 UTC m=+51.092968341"
Nov 04 03:17:16.737339 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:16.737311    2579 scope.go:122] "RemoveContainer" containerID="9075272d4964f13dc75390adcbd26ac4ef501c2d09e16c73075206e396a8a4b3"
Nov 04 03:17:17.042816 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:17.042786    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-p9989_068619f8-7fbb-4b22-ac7d-a1f144e084c0/console-operator/2.log"
Nov 04 03:17:17.043286 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:17.043263    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-p9989_068619f8-7fbb-4b22-ac7d-a1f144e084c0/console-operator/1.log"
Nov 04 03:17:17.043445 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:17.043310    2579 generic.go:356] "Generic (PLEG): container finished" podID="068619f8-7fbb-4b22-ac7d-a1f144e084c0" containerID="92d2557ce0ed61ad88da7d5c6d76d645bf407cd2ce7733bd5f621f087bce4578" exitCode=255
Nov 04 03:17:17.043445 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:17.043361    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-p9989" event={"ID":"068619f8-7fbb-4b22-ac7d-a1f144e084c0","Type":"ContainerDied","Data":"92d2557ce0ed61ad88da7d5c6d76d645bf407cd2ce7733bd5f621f087bce4578"}
Nov 04 03:17:17.043577 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:17.043441    2579 scope.go:122] "RemoveContainer" containerID="9075272d4964f13dc75390adcbd26ac4ef501c2d09e16c73075206e396a8a4b3"
Nov 04 03:17:17.043845 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:17.043825    2579 scope.go:122] "RemoveContainer" containerID="92d2557ce0ed61ad88da7d5c6d76d645bf407cd2ce7733bd5f621f087bce4578"
Nov 04 03:17:17.044031 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:17.044008    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-p9989_openshift-console-operator(068619f8-7fbb-4b22-ac7d-a1f144e084c0)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-p9989" podUID="068619f8-7fbb-4b22-ac7d-a1f144e084c0"
Nov 04 03:17:18.052491 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:18.052443    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-tlhng" event={"ID":"d731646d-23ab-42d7-ae48-974929b3d2dc","Type":"ContainerStarted","Data":"9d3fca05f60d9114c1eed89d0c5b187c8e62a74956eb91345693fbdf214de748"}
Nov 04 03:17:18.053982 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:18.053958    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-p9989_068619f8-7fbb-4b22-ac7d-a1f144e084c0/console-operator/2.log"
Nov 04 03:17:18.160914 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:18.160863    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-86c478bb88-tlhng" podStartSLOduration=379.003165105 podStartE2EDuration="6m21.160847861s" totalImagesPullingTime="2.157682756s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:10:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:17:18.160469497 +0000 UTC m=+52.989442030" watchObservedRunningTime="2026-11-04 03:17:18.160847861 +0000 UTC m=+52.989820393"
Nov 04 03:17:19.051675 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:19.051637    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-p9989"
Nov 04 03:17:19.051850 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:19.051694    2579 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-p9989"
Nov 04 03:17:19.052014 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:19.052002    2579 scope.go:122] "RemoveContainer" containerID="92d2557ce0ed61ad88da7d5c6d76d645bf407cd2ce7733bd5f621f087bce4578"
Nov 04 03:17:19.052228 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:19.052209    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-p9989_openshift-console-operator(068619f8-7fbb-4b22-ac7d-a1f144e084c0)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-p9989" podUID="068619f8-7fbb-4b22-ac7d-a1f144e084c0"
Nov 04 03:17:19.062344 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:19.062303    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-s4852" event={"ID":"06698c23-887e-4ee5-9728-04098200c13d","Type":"ContainerStarted","Data":"07a4e9e1035329f12e8e12345611b87e2e05e5ff81646c65d8befe554fc4015f"}
Nov 04 03:17:19.062743 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:19.062377    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-s4852" event={"ID":"06698c23-887e-4ee5-9728-04098200c13d","Type":"ContainerStarted","Data":"66bcc963e19d615e9d7d6fb7c95af5106e816f554a149d01f03956139be3ecb7"}
Nov 04 03:17:19.062743 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:19.062537    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-s4852"
Nov 04 03:17:19.272789 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:19.272725    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-s4852" podStartSLOduration=18.291402363 podStartE2EDuration="21.272710928s" totalImagesPullingTime="2.981308565s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:16:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:17:19.271682221 +0000 UTC m=+54.100654754" watchObservedRunningTime="2026-11-04 03:17:19.272710928 +0000 UTC m=+54.101683459"
Nov 04 03:17:21.583273 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:21.583176    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/536a0057-a438-4ad9-a4cd-b26cb7f3c645-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-46btb\" (UID: \"536a0057-a438-4ad9-a4cd-b26cb7f3c645\") " pod="openshift-insights/insights-runtime-extractor-46btb"
Nov 04 03:17:21.583273 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:21.583273    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:21.583795 ip-10-0-128-50 kubenswrapper[2579]: E1104 03:17:21.583337    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/536a0057-a438-4ad9-a4cd-b26cb7f3c645-insights-runtime-extractor-tls podName:536a0057-a438-4ad9-a4cd-b26cb7f3c645 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:37.583324468 +0000 UTC m=+72.412296977 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/536a0057-a438-4ad9-a4cd-b26cb7f3c645-insights-runtime-extractor-tls") pod "insights-runtime-extractor-46btb" (UID: "536a0057-a438-4ad9-a4cd-b26cb7f3c645") : secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:21.583795 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:21.583331    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/536a0057-a438-4ad9-a4cd-b26cb7f3c645-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-46btb\" (UID: \"536a0057-a438-4ad9-a4cd-b26cb7f3c645\") " pod="openshift-insights/insights-runtime-extractor-46btb"
Nov 04 03:17:21.583795 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:21.583785    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/536a0057-a438-4ad9-a4cd-b26cb7f3c645-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-46btb\" (UID: \"536a0057-a438-4ad9-a4cd-b26cb7f3c645\") " pod="openshift-insights/insights-runtime-extractor-46btb"
Nov 04 03:17:23.933207 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:23.933178    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:29.067249 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:29.067216    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-s4852"
Nov 04 03:17:30.661672 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:30.661635    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc3d3c0e-2877-4264-93a7-be3756d2dc5c-metrics-certs\") pod \"network-metrics-daemon-lngrr\" (UID: \"fc3d3c0e-2877-4264-93a7-be3756d2dc5c\") " pod="openshift-multus/network-metrics-daemon-lngrr"
Nov 04 03:17:30.662170 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:30.661684    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1b21c41-4b51-4da8-8920-c38cf49554bf-metrics-certs\") pod \"router-default-7cfb798487-kbfzk\" (UID: \"c1b21c41-4b51-4da8-8920-c38cf49554bf\") " pod="openshift-ingress/router-default-7cfb798487-kbfzk"
Nov 04 03:17:30.662170 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:30.661725    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e487dae4-8ac9-4acd-a8ad-a0d149909475-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-4z8fp\" (UID: \"e487dae4-8ac9-4acd-a8ad-a0d149909475\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-4z8fp"
Nov 04 03:17:30.662170 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:30.661763    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e87dded5-1e1b-45cb-ab2e-ae12550dfca7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-gmt9g\" (UID: \"e87dded5-1e1b-45cb-ab2e-ae12550dfca7\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmt9g"
Nov 04 03:17:30.662170 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:30.661783    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d10e9268-df07-428b-ae25-528d9fe075d2-original-pull-secret\") pod \"global-pull-secret-syncer-tkkhk\" (UID: \"d10e9268-df07-428b-ae25-528d9fe075d2\") " pod="kube-system/global-pull-secret-syncer-tkkhk"
Nov 04 03:17:30.662170 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:30.661800    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1b21c41-4b51-4da8-8920-c38cf49554bf-service-ca-bundle\") pod \"router-default-7cfb798487-kbfzk\" (UID: \"c1b21c41-4b51-4da8-8920-c38cf49554bf\") " pod="openshift-ingress/router-default-7cfb798487-kbfzk"
Nov 04 03:17:30.662598 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:30.662573    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1b21c41-4b51-4da8-8920-c38cf49554bf-service-ca-bundle\") pod \"router-default-7cfb798487-kbfzk\" (UID: \"c1b21c41-4b51-4da8-8920-c38cf49554bf\") " pod="openshift-ingress/router-default-7cfb798487-kbfzk"
Nov 04 03:17:30.664185 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:30.664162    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e487dae4-8ac9-4acd-a8ad-a0d149909475-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-4z8fp\" (UID: \"e487dae4-8ac9-4acd-a8ad-a0d149909475\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-4z8fp"
Nov 04 03:17:30.664275 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:30.664193    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1b21c41-4b51-4da8-8920-c38cf49554bf-metrics-certs\") pod \"router-default-7cfb798487-kbfzk\" (UID: \"c1b21c41-4b51-4da8-8920-c38cf49554bf\") " pod="openshift-ingress/router-default-7cfb798487-kbfzk"
Nov 04 03:17:30.664275 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:30.664237    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e87dded5-1e1b-45cb-ab2e-ae12550dfca7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-gmt9g\" (UID: \"e87dded5-1e1b-45cb-ab2e-ae12550dfca7\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmt9g"
Nov 04 03:17:30.756785 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:30.756738    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 04 03:17:30.756785 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:30.756769    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 04 03:17:30.765161 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:30.765129    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc3d3c0e-2877-4264-93a7-be3756d2dc5c-metrics-certs\") pod \"network-metrics-daemon-lngrr\" (UID: \"fc3d3c0e-2877-4264-93a7-be3756d2dc5c\") " pod="openshift-multus/network-metrics-daemon-lngrr"
Nov 04 03:17:30.765161 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:30.765156    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d10e9268-df07-428b-ae25-528d9fe075d2-original-pull-secret\") pod \"global-pull-secret-syncer-tkkhk\" (UID: \"d10e9268-df07-428b-ae25-528d9fe075d2\") " pod="kube-system/global-pull-secret-syncer-tkkhk"
Nov 04 03:17:30.833805 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:30.833754    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmt9g"
Nov 04 03:17:30.856184 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:30.856150    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7cfb798487-kbfzk"
Nov 04 03:17:30.865124 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:30.865092    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-4z8fp"
Nov 04 03:17:30.955808 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:30.955780    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-zsv6g\""
Nov 04 03:17:30.958067 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:30.957865    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkkhk"
Nov 04 03:17:30.970194 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:30.969971    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xwzp4\""
Nov 04 03:17:30.971302 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:30.971272    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lngrr"
Nov 04 03:17:31.242695 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:31.242664    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-gmt9g"]
Nov 04 03:17:31.249962 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:17:31.249930    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode87dded5_1e1b_45cb_ab2e_ae12550dfca7.slice/crio-14e9b215aa751f29082b433860d3ce4732d3e20959d5580e3a34f58f006c3ad4 WatchSource:0}: Error finding container 14e9b215aa751f29082b433860d3ce4732d3e20959d5580e3a34f58f006c3ad4: Status 404 returned error can't find the container with id 14e9b215aa751f29082b433860d3ce4732d3e20959d5580e3a34f58f006c3ad4
Nov 04 03:17:31.289081 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:31.289026    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-4z8fp"]
Nov 04 03:17:31.296873 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:31.296841    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7cfb798487-kbfzk"]
Nov 04 03:17:31.303340 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:17:31.303314    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1b21c41_4b51_4da8_8920_c38cf49554bf.slice/crio-b89d7825ec0f0d5bdf3487d8703a80adb02388b74aaf54dffbd2b094acbf119d WatchSource:0}: Error finding container b89d7825ec0f0d5bdf3487d8703a80adb02388b74aaf54dffbd2b094acbf119d: Status 404 returned error can't find the container with id b89d7825ec0f0d5bdf3487d8703a80adb02388b74aaf54dffbd2b094acbf119d
Nov 04 03:17:31.355551 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:31.355516    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tkkhk"]
Nov 04 03:17:31.361150 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:17:31.361120    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd10e9268_df07_428b_ae25_528d9fe075d2.slice/crio-a2c6ca82d9cc8cea0f09fc520c8c7d3bbc5ff0074d0c828f3a908af7bf5e7cac WatchSource:0}: Error finding container a2c6ca82d9cc8cea0f09fc520c8c7d3bbc5ff0074d0c828f3a908af7bf5e7cac: Status 404 returned error can't find the container with id a2c6ca82d9cc8cea0f09fc520c8c7d3bbc5ff0074d0c828f3a908af7bf5e7cac
Nov 04 03:17:31.363654 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:31.363628    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lngrr"]
Nov 04 03:17:31.370239 ip-10-0-128-50 kubenswrapper[2579]: W1104 03:17:31.370211    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc3d3c0e_2877_4264_93a7_be3756d2dc5c.slice/crio-ad7ea138aa3469e914d4fef4e4b959583c82867a03c59c662cd027e8a5b163ae WatchSource:0}: Error finding container ad7ea138aa3469e914d4fef4e4b959583c82867a03c59c662cd027e8a5b163ae: Status 404 returned error can't find the container with id ad7ea138aa3469e914d4fef4e4b959583c82867a03c59c662cd027e8a5b163ae
Nov 04 03:17:31.665862 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:31.665783    2579 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7cb9f7c89d-wnpll"]
Nov 04 03:17:31.671756 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:31.671705    2579 patch_prober.go:29] interesting pod/image-registry-7cb9f7c89d-wnpll container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Nov 04 03:17:31.671904 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:31.671774    2579 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll" podUID="fc612506-b699-44bc-a81b-06fee4e819fe" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 03:17:32.104394 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:32.104351    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7cfb798487-kbfzk" event={"ID":"c1b21c41-4b51-4da8-8920-c38cf49554bf","Type":"ContainerStarted","Data":"07ff45813c9618d9866b2293ae7b5de4a8f6fdbc8304688ed98c353d30555cfc"}
Nov 04 03:17:32.104593 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:32.104436    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7cfb798487-kbfzk" event={"ID":"c1b21c41-4b51-4da8-8920-c38cf49554bf","Type":"ContainerStarted","Data":"b89d7825ec0f0d5bdf3487d8703a80adb02388b74aaf54dffbd2b094acbf119d"}
Nov 04 03:17:32.105662 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:32.105630    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lngrr" event={"ID":"fc3d3c0e-2877-4264-93a7-be3756d2dc5c","Type":"ContainerStarted","Data":"ad7ea138aa3469e914d4fef4e4b959583c82867a03c59c662cd027e8a5b163ae"}
Nov 04 03:17:32.106794 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:32.106767    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tkkhk" event={"ID":"d10e9268-df07-428b-ae25-528d9fe075d2","Type":"ContainerStarted","Data":"a2c6ca82d9cc8cea0f09fc520c8c7d3bbc5ff0074d0c828f3a908af7bf5e7cac"}
Nov 04 03:17:32.107921 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:32.107894    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-4z8fp" event={"ID":"e487dae4-8ac9-4acd-a8ad-a0d149909475","Type":"ContainerStarted","Data":"a0039a17eb541c7e5cc28be3b894d6d1cefd9218af0ba5c89030e7eea937627a"}
Nov 04 03:17:32.109012 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:32.108994    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmt9g" event={"ID":"e87dded5-1e1b-45cb-ab2e-ae12550dfca7","Type":"ContainerStarted","Data":"14e9b215aa751f29082b433860d3ce4732d3e20959d5580e3a34f58f006c3ad4"}
Nov 04 03:17:32.217704 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:32.217646    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/router-default-7cfb798487-kbfzk" podStartSLOduration=442.217631367 podStartE2EDuration="7m22.217631367s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:10:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:17:32.217359672 +0000 UTC m=+67.046332204" watchObservedRunningTime="2026-11-04 03:17:32.217631367 +0000 UTC m=+67.046603899"
Nov 04 03:17:32.857374 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:32.857340    2579 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7cfb798487-kbfzk"
Nov 04 03:17:32.860611 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:32.860587    2579 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7cfb798487-kbfzk"
Nov 04 03:17:33.117206 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:33.117079    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7cfb798487-kbfzk"
Nov 04 03:17:33.118450 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:33.118183    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7cfb798487-kbfzk"
Nov 04 03:17:34.123818 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:34.123774    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lngrr" event={"ID":"fc3d3c0e-2877-4264-93a7-be3756d2dc5c","Type":"ContainerStarted","Data":"745c2d400c9116cb3bb71cf05aa21da75e607500aec4b9553508e712f8c5fe13"}
Nov 04 03:17:34.124209 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:34.123837    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lngrr" event={"ID":"fc3d3c0e-2877-4264-93a7-be3756d2dc5c","Type":"ContainerStarted","Data":"bb281469f9e00eba0f8f6273d99e460ae58cd4370359205a2d09c464bf57a580"}
Nov 04 03:17:34.232193 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:34.232141    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-lngrr" podStartSLOduration=67.235127256 podStartE2EDuration="1m9.232126228s" totalImagesPullingTime="1.996998972s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:16:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:17:34.232036939 +0000 UTC m=+69.061009471" watchObservedRunningTime="2026-11-04 03:17:34.232126228 +0000 UTC m=+69.061098759"
Nov 04 03:17:34.647564 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:34.647529    2579 patch_prober.go:29] interesting pod/image-registry-79d6dd87ff-n2jnc container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Nov 04 03:17:34.647736 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:34.647582    2579 prober.go:122] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc" podUID="f1f61b42-3b07-4892-90a7-2efa024e435b" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 03:17:35.130779 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:35.130267    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-4z8fp" event={"ID":"e487dae4-8ac9-4acd-a8ad-a0d149909475","Type":"ContainerStarted","Data":"7040c02a6ed03eb243ff7adb0dfe90c542fb1e80a7ff602779687514061bb2ed"}
Nov 04 03:17:35.130779 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:35.130327    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-4z8fp" event={"ID":"e487dae4-8ac9-4acd-a8ad-a0d149909475","Type":"ContainerStarted","Data":"61cb1bda92e1f91e5878bbc71e9a780a4109c8ddef5a2ca97789340b33513ef2"}
Nov 04 03:17:35.132145 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:35.132119    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmt9g" event={"ID":"e87dded5-1e1b-45cb-ab2e-ae12550dfca7","Type":"ContainerStarted","Data":"470195ddd74124ddd199632e4bc06f60f01748b64c98733b40b16d67c0dd32f3"}
Nov 04 03:17:35.241834 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:35.241772    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-4z8fp" podStartSLOduration=439.495525052 podStartE2EDuration="7m22.241755084s" totalImagesPullingTime="2.746230032s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:10:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:17:35.240467201 +0000 UTC m=+70.069439736" watchObservedRunningTime="2026-11-04 03:17:35.241755084 +0000 UTC m=+70.070727616"
Nov 04 03:17:35.350931 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:35.350874    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmt9g" podStartSLOduration=444.478540026 podStartE2EDuration="7m27.350852609s" totalImagesPullingTime="2.872312583s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:10:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:17:35.349642362 +0000 UTC m=+70.178614895" watchObservedRunningTime="2026-11-04 03:17:35.350852609 +0000 UTC m=+70.179825142"
Nov 04 03:17:37.048846 ip-10-0-128-50 kubenswrapper[2579]: I1104 03:17:37.048813    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:17:37.360879 ip-10-0-128-50 systemd[1]: Stopping Kubernetes Kubelet...
Nov 04 03:17:37.369026 ip-10-0-128-50 systemd[1]: kubelet.service: Deactivated successfully.
Nov 04 03:17:37.369213 ip-10-0-128-50 systemd[1]: Stopped Kubernetes Kubelet.
Nov 04 03:17:37.369393 ip-10-0-128-50 systemd[1]: kubelet.service: Consumed 4.397s CPU time.
Nov 04 03:17:37.389317 ip-10-0-128-50 systemd[1]: Starting Kubernetes Kubelet...
Nov 04 03:17:37.453077 ip-10-0-128-50 kubenswrapper[5609]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 03:17:37.453077 ip-10-0-128-50 kubenswrapper[5609]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 04 03:17:37.453077 ip-10-0-128-50 kubenswrapper[5609]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 03:17:37.453077 ip-10-0-128-50 kubenswrapper[5609]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 03:17:37.455217 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455124    5609 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:17:37.455217 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455218    5609 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:17:37.455287 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455222    5609 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:17:37.455287 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455226    5609 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:17:37.455287 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455228    5609 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:17:37.455287 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455232    5609 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:17:37.455287 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455235    5609 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:17:37.455287 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455238    5609 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:17:37.455287 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455240    5609 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:17:37.455287 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455243    5609 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:17:37.455287 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455246    5609 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:17:37.455287 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455248    5609 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:17:37.455287 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455251    5609 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:17:37.455287 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455254    5609 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:17:37.455287 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455256    5609 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:17:37.455287 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455259    5609 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:17:37.455287 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455261    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:17:37.455287 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455264    5609 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:17:37.455287 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455267    5609 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:17:37.455287 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455269    5609 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:17:37.455287 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455271    5609 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:17:37.455787 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455275    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:17:37.455787 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455277    5609 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:17:37.455787 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455282    5609 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:17:37.455787 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455287    5609 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:17:37.455787 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455291    5609 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:17:37.455787 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455300    5609 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:17:37.455787 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455303    5609 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:17:37.455787 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455306    5609 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:17:37.455787 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455309    5609 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:17:37.455787 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455311    5609 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:17:37.455787 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455314    5609 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:17:37.455787 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455316    5609 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:17:37.455787 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455319    5609 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:17:37.455787 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455322    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:17:37.455787 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455324    5609 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:17:37.455787 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455327    5609 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:17:37.455787 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455329    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:17:37.455787 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455332    5609 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:17:37.455787 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455334    5609 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:17:37.456247 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455337    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:17:37.456247 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455339    5609 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:17:37.456247 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455342    5609 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:17:37.456247 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455344    5609 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:17:37.456247 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455347    5609 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:17:37.456247 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455349    5609 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:17:37.456247 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455352    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:17:37.456247 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455355    5609 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:17:37.456247 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455357    5609 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:17:37.456247 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455360    5609 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:17:37.456247 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455363    5609 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:17:37.456247 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455365    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:17:37.456247 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455368    5609 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:17:37.456247 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455372    5609 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:17:37.456247 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455374    5609 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:17:37.456247 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455377    5609 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:17:37.456247 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455379    5609 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:17:37.456247 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455382    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:17:37.456247 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455385    5609 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:17:37.456247 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455388    5609 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:17:37.456760 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455391    5609 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:17:37.456760 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455393    5609 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:17:37.456760 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455414    5609 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:17:37.456760 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455419    5609 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:17:37.456760 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455424    5609 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:17:37.456760 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455428    5609 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:17:37.456760 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455432    5609 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:17:37.456760 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455437    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:17:37.456760 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455441    5609 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:17:37.456760 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455445    5609 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:17:37.456760 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455448    5609 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:17:37.456760 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455451    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:17:37.456760 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455453    5609 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:17:37.456760 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455456    5609 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:17:37.456760 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455459    5609 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:17:37.456760 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455461    5609 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:17:37.456760 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455466    5609 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:17:37.456760 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455468    5609 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:17:37.456760 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455471    5609 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:17:37.456760 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455474    5609 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:17:37.457257 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455476    5609 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:17:37.457257 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455479    5609 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:17:37.457257 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455482    5609 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:17:37.457257 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455484    5609 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:17:37.457257 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455487    5609 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:17:37.457257 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455491    5609 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:17:37.457257 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455494    5609 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:17:37.457257 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455497    5609 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:17:37.457257 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455499    5609 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:17:37.457257 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455503    5609 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:17:37.457257 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455507    5609 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:17:37.457257 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455510    5609 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:17:37.457257 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455514    5609 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:17:37.457257 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455517    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:17:37.457257 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455519    5609 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:17:37.457257 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455522    5609 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:17:37.457257 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455526    5609 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:17:37.457257 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455529    5609 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:17:37.457257 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455532    5609 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:17:37.457768 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455534    5609 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:17:37.457768 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455537    5609 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:17:37.457768 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455540    5609 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:17:37.457768 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455543    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:17:37.457768 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455545    5609 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:17:37.457768 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455548    5609 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:17:37.457768 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.455551    5609 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:17:37.457768 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456330    5609 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:17:37.457768 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456338    5609 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:17:37.457768 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456341    5609 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:17:37.457768 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456343    5609 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:17:37.457768 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456346    5609 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:17:37.457768 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456349    5609 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:17:37.457768 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456352    5609 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:17:37.457768 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456355    5609 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:17:37.457768 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456357    5609 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:17:37.457768 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456360    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:17:37.457768 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456363    5609 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:17:37.457768 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456366    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:17:37.458255 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456368    5609 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:17:37.458255 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456371    5609 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:17:37.458255 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456374    5609 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:17:37.458255 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456376    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:17:37.458255 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456379    5609 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:17:37.458255 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456382    5609 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:17:37.458255 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456384    5609 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:17:37.458255 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456388    5609 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:17:37.458255 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456391    5609 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:17:37.458255 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456393    5609 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:17:37.458255 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456413    5609 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:17:37.458255 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456418    5609 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:17:37.458255 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456422    5609 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:17:37.458255 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456426    5609 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:17:37.458255 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456430    5609 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:17:37.458255 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456434    5609 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:17:37.458255 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456438    5609 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:17:37.458255 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456441    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:17:37.458255 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456444    5609 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:17:37.458255 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456447    5609 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:17:37.458807 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456450    5609 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:17:37.458807 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456452    5609 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:17:37.458807 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456455    5609 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:17:37.458807 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456458    5609 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:17:37.458807 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456460    5609 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:17:37.458807 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456463    5609 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:17:37.458807 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456465    5609 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:17:37.458807 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456468    5609 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:17:37.458807 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456471    5609 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:17:37.458807 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456473    5609 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:17:37.458807 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456476    5609 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:17:37.458807 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456488    5609 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:17:37.458807 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456491    5609 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:17:37.458807 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456494    5609 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:17:37.458807 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456496    5609 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:17:37.458807 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456499    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:17:37.458807 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456502    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:17:37.458807 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456505    5609 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:17:37.458807 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456508    5609 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:17:37.459290 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456511    5609 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:17:37.459290 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456519    5609 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:17:37.459290 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456522    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:17:37.459290 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456525    5609 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:17:37.459290 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456528    5609 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:17:37.459290 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456530    5609 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:17:37.459290 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456533    5609 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:17:37.459290 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456536    5609 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:17:37.459290 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456539    5609 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:17:37.459290 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456541    5609 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:17:37.459290 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456544    5609 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:17:37.459290 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456547    5609 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:17:37.459290 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456549    5609 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:17:37.459290 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456552    5609 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:17:37.459290 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456554    5609 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:17:37.459290 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456557    5609 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:17:37.459290 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456561    5609 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:17:37.459290 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456566    5609 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:17:37.459290 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456570    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:17:37.459777 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456573    5609 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:17:37.459777 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456576    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:17:37.459777 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456578    5609 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:17:37.459777 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456581    5609 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:17:37.459777 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456584    5609 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:17:37.459777 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456587    5609 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:17:37.459777 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456590    5609 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:17:37.459777 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456593    5609 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:17:37.459777 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456596    5609 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:17:37.459777 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456599    5609 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:17:37.459777 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456601    5609 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:17:37.459777 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456604    5609 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:17:37.459777 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456606    5609 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:17:37.459777 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456609    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:17:37.459777 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456611    5609 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:17:37.459777 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456621    5609 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:17:37.459777 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456625    5609 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:17:37.459777 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456628    5609 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:17:37.459777 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456632    5609 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:17:37.459777 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456634    5609 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:17:37.460264 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456638    5609 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:17:37.460264 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456640    5609 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:17:37.460264 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456644    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:17:37.460264 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456647    5609 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:17:37.460264 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456649    5609 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:17:37.460264 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456652    5609 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:17:37.460264 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456656    5609 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:17:37.460264 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456659    5609 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:17:37.460264 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456661    5609 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:17:37.460264 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456664    5609 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:17:37.460264 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456666    5609 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:17:37.460264 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456669    5609 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:17:37.460264 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456672    5609 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:17:37.460264 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456674    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:17:37.460264 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456681    5609 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:17:37.460264 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.456684    5609 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:17:37.460264 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457215    5609 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:17:37.460264 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457221    5609 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:17:37.460264 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457224    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:17:37.460746 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457227    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:17:37.460746 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457229    5609 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:17:37.460746 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457232    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:17:37.460746 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457235    5609 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:17:37.460746 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457238    5609 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:17:37.460746 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457241    5609 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:17:37.460746 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457243    5609 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:17:37.460746 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457247    5609 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:17:37.460746 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457250    5609 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:17:37.460746 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457252    5609 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:17:37.460746 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457255    5609 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:17:37.460746 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457258    5609 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:17:37.460746 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457261    5609 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:17:37.460746 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457264    5609 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:17:37.460746 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457266    5609 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:17:37.460746 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457270    5609 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:17:37.460746 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457273    5609 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:17:37.460746 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457276    5609 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:17:37.460746 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457280    5609 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:17:37.461240 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457284    5609 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:17:37.461240 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457287    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:17:37.461240 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457290    5609 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:17:37.461240 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457292    5609 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:17:37.461240 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457295    5609 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:17:37.461240 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457297    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:17:37.461240 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457300    5609 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:17:37.461240 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457302    5609 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:17:37.461240 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457306    5609 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:17:37.461240 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457309    5609 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:17:37.461240 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457312    5609 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:17:37.461240 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457314    5609 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:17:37.461240 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457317    5609 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:17:37.461240 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457319    5609 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:17:37.461240 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457322    5609 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:17:37.461240 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457324    5609 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:17:37.461240 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457327    5609 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:17:37.461240 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457329    5609 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:17:37.461240 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457332    5609 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:17:37.461240 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457334    5609 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:17:37.461725 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457337    5609 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:17:37.461725 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457340    5609 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:17:37.461725 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457343    5609 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:17:37.461725 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457345    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:17:37.461725 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457348    5609 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:17:37.461725 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457351    5609 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:17:37.461725 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457354    5609 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:17:37.461725 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457356    5609 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:17:37.461725 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457359    5609 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:17:37.461725 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457362    5609 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:17:37.461725 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457365    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:17:37.461725 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457368    5609 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:17:37.461725 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457370    5609 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:17:37.461725 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457373    5609 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:17:37.461725 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457376    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:17:37.461725 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457378    5609 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:17:37.461725 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457381    5609 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:17:37.461725 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457384    5609 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:17:37.461725 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457386    5609 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:17:37.462184 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457389    5609 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:17:37.462184 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457392    5609 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:17:37.462184 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457395    5609 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:17:37.462184 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457412    5609 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:17:37.462184 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457417    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:17:37.462184 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457421    5609 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:17:37.462184 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457425    5609 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:17:37.462184 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457430    5609 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:17:37.462184 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457434    5609 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:17:37.462184 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457437    5609 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:17:37.462184 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457440    5609 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:17:37.462184 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457442    5609 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:17:37.462184 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457446    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:17:37.462184 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457448    5609 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:17:37.462184 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457451    5609 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:17:37.462184 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457454    5609 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:17:37.462184 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457458    5609 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:17:37.462184 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457460    5609 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:17:37.462184 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457463    5609 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:17:37.462184 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457466    5609 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:17:37.462724 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457468    5609 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:17:37.462724 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457471    5609 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:17:37.462724 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457473    5609 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:17:37.462724 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457476    5609 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:17:37.462724 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457479    5609 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:17:37.462724 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457481    5609 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:17:37.462724 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457484    5609 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:17:37.462724 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457486    5609 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:17:37.462724 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457489    5609 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:17:37.462724 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457492    5609 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:17:37.462724 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457494    5609 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:17:37.462724 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457497    5609 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:17:37.462724 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457500    5609 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:17:37.462724 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457502    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:17:37.462724 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457505    5609 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:17:37.462724 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457508    5609 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:17:37.462724 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457510    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:17:37.462724 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457513    5609 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:17:37.462724 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457515    5609 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:17:37.462724 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457518    5609 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:17:37.463211 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457522    5609 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:17:37.463211 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457524    5609 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:17:37.463211 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457527    5609 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:17:37.463211 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457529    5609 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:17:37.463211 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.457532    5609 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:17:37.463211 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457641    5609 flags.go:64] FLAG: --address="0.0.0.0"
Nov 04 03:17:37.463211 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457655    5609 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 04 03:17:37.463211 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457663    5609 flags.go:64] FLAG: --anonymous-auth="true"
Nov 04 03:17:37.463211 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457668    5609 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 04 03:17:37.463211 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457673    5609 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 04 03:17:37.463211 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457677    5609 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 04 03:17:37.463211 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457681    5609 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 04 03:17:37.463211 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457687    5609 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 04 03:17:37.463211 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457690    5609 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 04 03:17:37.463211 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457694    5609 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 04 03:17:37.463211 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457697    5609 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 04 03:17:37.463211 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457701    5609 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 04 03:17:37.463211 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457705    5609 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 04 03:17:37.463211 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457708    5609 flags.go:64] FLAG: --cgroup-root=""
Nov 04 03:17:37.463211 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457711    5609 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 04 03:17:37.463211 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457714    5609 flags.go:64] FLAG: --client-ca-file=""
Nov 04 03:17:37.463211 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457717    5609 flags.go:64] FLAG: --cloud-provider="external"
Nov 04 03:17:37.463211 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457720    5609 flags.go:64] FLAG: --cluster-dns="[]"
Nov 04 03:17:37.464029 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457725    5609 flags.go:64] FLAG: --cluster-domain=""
Nov 04 03:17:37.464029 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457728    5609 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 04 03:17:37.464029 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457731    5609 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 04 03:17:37.464029 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457734    5609 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 04 03:17:37.464029 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457737    5609 flags.go:64] FLAG: --container-log-max-files="5"
Nov 04 03:17:37.464029 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457741    5609 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 04 03:17:37.464029 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457744    5609 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 04 03:17:37.464029 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457747    5609 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 04 03:17:37.464029 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457753    5609 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 04 03:17:37.464029 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457756    5609 flags.go:64] FLAG: --contention-profiling="false"
Nov 04 03:17:37.464029 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457759    5609 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 04 03:17:37.464029 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457762    5609 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 04 03:17:37.464029 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457765    5609 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 04 03:17:37.464029 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457769    5609 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 04 03:17:37.464029 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457773    5609 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 04 03:17:37.464029 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457777    5609 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 04 03:17:37.464029 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457780    5609 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 04 03:17:37.464029 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457783    5609 flags.go:64] FLAG: --enable-load-reader="false"
Nov 04 03:17:37.464029 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457786    5609 flags.go:64] FLAG: --enable-server="true"
Nov 04 03:17:37.464029 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457789    5609 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 04 03:17:37.464029 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457794    5609 flags.go:64] FLAG: --event-burst="100"
Nov 04 03:17:37.464029 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457799    5609 flags.go:64] FLAG: --event-qps="50"
Nov 04 03:17:37.464029 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457803    5609 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 04 03:17:37.464029 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457806    5609 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 04 03:17:37.464892 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457809    5609 flags.go:64] FLAG: --eviction-hard=""
Nov 04 03:17:37.464892 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457813    5609 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 04 03:17:37.464892 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457816    5609 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 04 03:17:37.464892 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457819    5609 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 04 03:17:37.464892 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457822    5609 flags.go:64] FLAG: --eviction-soft=""
Nov 04 03:17:37.464892 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457826    5609 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 04 03:17:37.464892 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457828    5609 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 04 03:17:37.464892 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457832    5609 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 04 03:17:37.464892 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457835    5609 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 04 03:17:37.464892 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457838    5609 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 04 03:17:37.464892 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457841    5609 flags.go:64] FLAG: --fail-swap-on="true"
Nov 04 03:17:37.464892 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457844    5609 flags.go:64] FLAG: --feature-gates=""
Nov 04 03:17:37.464892 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457848    5609 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 04 03:17:37.464892 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457852    5609 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 04 03:17:37.464892 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457855    5609 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 04 03:17:37.464892 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457858    5609 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 04 03:17:37.464892 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457863    5609 flags.go:64] FLAG: --healthz-port="10248"
Nov 04 03:17:37.464892 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457867    5609 flags.go:64] FLAG: --help="false"
Nov 04 03:17:37.464892 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457870    5609 flags.go:64] FLAG: --hostname-override="ip-10-0-128-50.us-west-2.compute.internal"
Nov 04 03:17:37.464892 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457874    5609 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 04 03:17:37.464892 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457877    5609 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 04 03:17:37.464892 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457880    5609 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 04 03:17:37.464892 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457883    5609 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 04 03:17:37.465610 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457886    5609 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 04 03:17:37.465610 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457890    5609 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 04 03:17:37.465610 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457893    5609 flags.go:64] FLAG: --image-service-endpoint=""
Nov 04 03:17:37.465610 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457896    5609 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 04 03:17:37.465610 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457899    5609 flags.go:64] FLAG: --kube-api-burst="100"
Nov 04 03:17:37.465610 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457902    5609 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 04 03:17:37.465610 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457907    5609 flags.go:64] FLAG: --kube-api-qps="50"
Nov 04 03:17:37.465610 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457910    5609 flags.go:64] FLAG: --kube-reserved=""
Nov 04 03:17:37.465610 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457913    5609 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 04 03:17:37.465610 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457916    5609 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 04 03:17:37.465610 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457920    5609 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 04 03:17:37.465610 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457923    5609 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 04 03:17:37.465610 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457926    5609 flags.go:64] FLAG: --lock-file=""
Nov 04 03:17:37.465610 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457929    5609 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 04 03:17:37.465610 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457932    5609 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 04 03:17:37.465610 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457935    5609 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 04 03:17:37.465610 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457940    5609 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 04 03:17:37.465610 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457943    5609 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 04 03:17:37.465610 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457946    5609 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 04 03:17:37.465610 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457949    5609 flags.go:64] FLAG: --logging-format="text"
Nov 04 03:17:37.465610 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457952    5609 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 04 03:17:37.465610 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457956    5609 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 04 03:17:37.465610 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457958    5609 flags.go:64] FLAG: --manifest-url=""
Nov 04 03:17:37.465610 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457962    5609 flags.go:64] FLAG: --manifest-url-header=""
Nov 04 03:17:37.465610 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457966    5609 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 04 03:17:37.466218 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457971    5609 flags.go:64] FLAG: --max-open-files="1000000"
Nov 04 03:17:37.466218 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457975    5609 flags.go:64] FLAG: --max-pods="110"
Nov 04 03:17:37.466218 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457978    5609 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 04 03:17:37.466218 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457981    5609 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 04 03:17:37.466218 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457984    5609 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 04 03:17:37.466218 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457987    5609 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 04 03:17:37.466218 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457990    5609 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 04 03:17:37.466218 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457993    5609 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 04 03:17:37.466218 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.457996    5609 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 04 03:17:37.466218 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458003    5609 flags.go:64] FLAG: --node-status-max-images="50"
Nov 04 03:17:37.466218 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458007    5609 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 04 03:17:37.466218 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458010    5609 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 04 03:17:37.466218 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458013    5609 flags.go:64] FLAG: --pod-cidr=""
Nov 04 03:17:37.466218 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458018    5609 flags.go:64] FLAG: --pod-manifest-path=""
Nov 04 03:17:37.466218 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458021    5609 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 04 03:17:37.466218 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458024    5609 flags.go:64] FLAG: --pods-per-core="0"
Nov 04 03:17:37.466218 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458027    5609 flags.go:64] FLAG: --port="10250"
Nov 04 03:17:37.466218 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458030    5609 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 04 03:17:37.466218 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458033    5609 flags.go:64] FLAG: --provider-id="aws:///us-west-2a/i-0bab39cfb9c95f313"
Nov 04 03:17:37.466218 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458036    5609 flags.go:64] FLAG: --qos-reserved=""
Nov 04 03:17:37.466218 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458039    5609 flags.go:64] FLAG: --read-only-port="10255"
Nov 04 03:17:37.466218 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458042    5609 flags.go:64] FLAG: --register-node="true"
Nov 04 03:17:37.466218 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458045    5609 flags.go:64] FLAG: --register-with-taints=""
Nov 04 03:17:37.466218 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458049    5609 flags.go:64] FLAG: --registry-burst="10"
Nov 04 03:17:37.466218 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458052    5609 flags.go:64] FLAG: --registry-qps="5"
Nov 04 03:17:37.466218 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458055    5609 flags.go:64] FLAG: --reserved-cpus=""
Nov 04 03:17:37.466872 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458058    5609 flags.go:64] FLAG: --reserved-memory=""
Nov 04 03:17:37.466872 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458062    5609 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 04 03:17:37.466872 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458066    5609 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 04 03:17:37.466872 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458069    5609 flags.go:64] FLAG: --rotate-certificates="false"
Nov 04 03:17:37.466872 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458072    5609 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 04 03:17:37.466872 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458075    5609 flags.go:64] FLAG: --runonce="false"
Nov 04 03:17:37.466872 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458080    5609 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 04 03:17:37.466872 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458083    5609 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 04 03:17:37.466872 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458086    5609 flags.go:64] FLAG: --seccomp-default="false"
Nov 04 03:17:37.466872 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458089    5609 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 04 03:17:37.466872 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458092    5609 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 04 03:17:37.466872 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458095    5609 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 04 03:17:37.466872 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458098    5609 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 04 03:17:37.466872 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458102    5609 flags.go:64] FLAG: --storage-driver-password="root"
Nov 04 03:17:37.466872 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458105    5609 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 04 03:17:37.466872 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458108    5609 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 04 03:17:37.466872 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458111    5609 flags.go:64] FLAG: --storage-driver-user="root"
Nov 04 03:17:37.466872 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458113    5609 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 04 03:17:37.466872 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458117    5609 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 04 03:17:37.466872 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458124    5609 flags.go:64] FLAG: --system-cgroups=""
Nov 04 03:17:37.466872 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458128    5609 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 04 03:17:37.466872 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458133    5609 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 04 03:17:37.466872 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458136    5609 flags.go:64] FLAG: --tls-cert-file=""
Nov 04 03:17:37.466872 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458139    5609 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 04 03:17:37.466872 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458143    5609 flags.go:64] FLAG: --tls-min-version=""
Nov 04 03:17:37.467507 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458146    5609 flags.go:64] FLAG: --tls-private-key-file=""
Nov 04 03:17:37.467507 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458149    5609 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 04 03:17:37.467507 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458152    5609 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 04 03:17:37.467507 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458155    5609 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 04 03:17:37.467507 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458158    5609 flags.go:64] FLAG: --v="2"
Nov 04 03:17:37.467507 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458162    5609 flags.go:64] FLAG: --version="false"
Nov 04 03:17:37.467507 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458167    5609 flags.go:64] FLAG: --vmodule=""
Nov 04 03:17:37.467507 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458171    5609 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 04 03:17:37.467507 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458174    5609 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 04 03:17:37.467507 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458329    5609 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:17:37.467507 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458333    5609 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:17:37.467507 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458337    5609 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:17:37.467507 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458340    5609 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:17:37.467507 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458345    5609 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:17:37.467507 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458349    5609 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:17:37.467507 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458351    5609 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:17:37.467507 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458354    5609 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:17:37.467507 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458357    5609 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:17:37.467507 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458360    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:17:37.467507 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458363    5609 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:17:37.467507 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458366    5609 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:17:37.467507 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458369    5609 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:17:37.468060 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458371    5609 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:17:37.468060 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458374    5609 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:17:37.468060 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458377    5609 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:17:37.468060 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458379    5609 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:17:37.468060 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458383    5609 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:17:37.468060 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458385    5609 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:17:37.468060 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458389    5609 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:17:37.468060 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458392    5609 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:17:37.468060 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458395    5609 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:17:37.468060 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458410    5609 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:17:37.468060 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458413    5609 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:17:37.468060 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458415    5609 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:17:37.468060 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458418    5609 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:17:37.468060 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458421    5609 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:17:37.468060 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458423    5609 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:17:37.468060 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458426    5609 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:17:37.468060 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458429    5609 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:17:37.468060 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458431    5609 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:17:37.468060 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458434    5609 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:17:37.468060 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458437    5609 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:17:37.468573 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458439    5609 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:17:37.468573 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458442    5609 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:17:37.468573 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458444    5609 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:17:37.468573 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458448    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:17:37.468573 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458451    5609 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:17:37.468573 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458454    5609 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:17:37.468573 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458457    5609 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:17:37.468573 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458459    5609 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:17:37.468573 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458462    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:17:37.468573 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458465    5609 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:17:37.468573 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458467    5609 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:17:37.468573 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458470    5609 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:17:37.468573 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458473    5609 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:17:37.468573 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458475    5609 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:17:37.468573 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458478    5609 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:17:37.468573 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458481    5609 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:17:37.468573 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458485    5609 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:17:37.468573 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458489    5609 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:17:37.468573 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458492    5609 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:17:37.469062 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458495    5609 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:17:37.469062 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458498    5609 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:17:37.469062 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458501    5609 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:17:37.469062 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458504    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:17:37.469062 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458506    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:17:37.469062 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458509    5609 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:17:37.469062 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458512    5609 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:17:37.469062 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458514    5609 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:17:37.469062 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458517    5609 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:17:37.469062 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458520    5609 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:17:37.469062 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458522    5609 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:17:37.469062 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458525    5609 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:17:37.469062 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458528    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:17:37.469062 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458530    5609 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:17:37.469062 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458533    5609 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:17:37.469062 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458536    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:17:37.469062 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458540    5609 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:17:37.469062 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458543    5609 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:17:37.469062 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458546    5609 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:17:37.469591 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458548    5609 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:17:37.469591 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458551    5609 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:17:37.469591 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458554    5609 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:17:37.469591 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458556    5609 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:17:37.469591 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458559    5609 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:17:37.469591 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458561    5609 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:17:37.469591 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458564    5609 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:17:37.469591 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458567    5609 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:17:37.469591 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458570    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:17:37.469591 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458572    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:17:37.469591 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458576    5609 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:17:37.469591 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458579    5609 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:17:37.469591 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458582    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:17:37.469591 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458584    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:17:37.469591 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458587    5609 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:17:37.469591 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458590    5609 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:17:37.469591 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458592    5609 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:17:37.469591 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458595    5609 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:17:37.469591 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458597    5609 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:17:37.470100 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458600    5609 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:17:37.470100 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458603    5609 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:17:37.470100 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458605    5609 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:17:37.470100 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458608    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:17:37.470100 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458611    5609 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:17:37.470100 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458614    5609 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:17:37.470100 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458616    5609 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:17:37.470100 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458619    5609 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:17:37.470100 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458621    5609 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:17:37.470100 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458624    5609 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:17:37.470100 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458628    5609 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:17:37.470100 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458631    5609 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:17:37.470100 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458633    5609 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:17:37.470100 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458636    5609 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:17:37.470100 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458640    5609 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:17:37.470100 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458644    5609 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:17:37.470569 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.458671    5609 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 03:17:37.470569 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458707    5609 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:17:37.470569 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458713    5609 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:17:37.470569 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458717    5609 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:17:37.470569 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458721    5609 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:17:37.470569 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458724    5609 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:17:37.470569 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458726    5609 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:17:37.470569 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458730    5609 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:17:37.470569 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458733    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:17:37.470569 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458737    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:17:37.470569 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458739    5609 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:17:37.470569 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458742    5609 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:17:37.470569 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458745    5609 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:17:37.470569 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458748    5609 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:17:37.470569 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458751    5609 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:17:37.470936 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458753    5609 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:17:37.470936 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458756    5609 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:17:37.470936 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458759    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:17:37.470936 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458761    5609 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:17:37.470936 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458764    5609 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:17:37.470936 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458766    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:17:37.470936 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458769    5609 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:17:37.470936 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458771    5609 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:17:37.470936 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458774    5609 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:17:37.470936 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458776    5609 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:17:37.470936 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458779    5609 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:17:37.470936 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458781    5609 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:17:37.470936 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458785    5609 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:17:37.470936 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458788    5609 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:17:37.470936 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458791    5609 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:17:37.470936 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458793    5609 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:17:37.470936 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458796    5609 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:17:37.470936 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458799    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:17:37.470936 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458802    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:17:37.470936 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458805    5609 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:17:37.471451 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458807    5609 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:17:37.471451 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458810    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:17:37.471451 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458826    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:17:37.471451 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458830    5609 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:17:37.471451 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458834    5609 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:17:37.471451 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458837    5609 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:17:37.471451 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458840    5609 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:17:37.471451 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458842    5609 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:17:37.471451 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458845    5609 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:17:37.471451 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458848    5609 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:17:37.471451 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458851    5609 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:17:37.471451 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458854    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:17:37.471451 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458857    5609 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:17:37.471451 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458859    5609 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:17:37.471451 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458862    5609 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:17:37.471451 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458865    5609 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:17:37.471451 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458867    5609 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:17:37.471451 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458870    5609 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:17:37.471451 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458872    5609 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:17:37.472037 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458875    5609 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:17:37.472037 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458878    5609 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:17:37.472037 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458880    5609 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:17:37.472037 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458885    5609 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:17:37.472037 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458889    5609 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:17:37.472037 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458893    5609 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:17:37.472037 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458896    5609 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:17:37.472037 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458898    5609 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:17:37.472037 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458901    5609 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:17:37.472037 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458904    5609 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:17:37.472037 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458906    5609 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:17:37.472037 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458909    5609 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:17:37.472037 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458912    5609 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:17:37.472037 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458915    5609 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:17:37.472037 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458917    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:17:37.472037 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458920    5609 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:17:37.472037 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458923    5609 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:17:37.472037 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458926    5609 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:17:37.472037 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458929    5609 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:17:37.472037 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458932    5609 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:17:37.472817 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458935    5609 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:17:37.472817 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458937    5609 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:17:37.472817 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458940    5609 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:17:37.472817 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458942    5609 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:17:37.472817 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458946    5609 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:17:37.472817 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458948    5609 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:17:37.472817 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458952    5609 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:17:37.472817 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458955    5609 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:17:37.472817 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458957    5609 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:17:37.472817 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458960    5609 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:17:37.472817 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458962    5609 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:17:37.472817 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458965    5609 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:17:37.472817 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458967    5609 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:17:37.472817 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458970    5609 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:17:37.472817 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458972    5609 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:17:37.472817 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458975    5609 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:17:37.472817 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458977    5609 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:17:37.472817 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458981    5609 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:17:37.472817 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458984    5609 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:17:37.472817 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458987    5609 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:17:37.473308 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458989    5609 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:17:37.473308 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458992    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:17:37.473308 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458994    5609 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:17:37.473308 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.458997    5609 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:17:37.473308 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.459000    5609 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:17:37.473308 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.459002    5609 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:17:37.473308 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.459005    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:17:37.473308 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.459007    5609 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:17:37.473308 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.459010    5609 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:17:37.473308 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.459014    5609 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:17:37.473308 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.459016    5609 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:17:37.473308 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.459019    5609 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:17:37.473308 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.459022    5609 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:17:37.473308 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.462854    5609 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 04 03:17:37.473308 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.462872    5609 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 04 03:17:37.473308 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.462892    5609 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 04 03:17:37.473308 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.462896    5609 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 04 03:17:37.473308 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.462971    5609 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:17:37.473308 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.462976    5609 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:17:37.473789 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.462979    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:17:37.473789 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.462982    5609 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:17:37.473789 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.462985    5609 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:17:37.473789 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.462988    5609 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:17:37.473789 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.462991    5609 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:17:37.473789 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.462994    5609 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:17:37.473789 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.462997    5609 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:17:37.473789 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463001    5609 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:17:37.473789 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463004    5609 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:17:37.473789 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463007    5609 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:17:37.473789 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463010    5609 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:17:37.473789 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463013    5609 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:17:37.473789 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463016    5609 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:17:37.473789 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463018    5609 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:17:37.473789 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463021    5609 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:17:37.473789 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463024    5609 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:17:37.473789 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463027    5609 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:17:37.473789 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463030    5609 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:17:37.473789 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463032    5609 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:17:37.473789 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463035    5609 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:17:37.474263 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463038    5609 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:17:37.474263 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463041    5609 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:17:37.474263 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463044    5609 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:17:37.474263 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463046    5609 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:17:37.474263 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463049    5609 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:17:37.474263 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463052    5609 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:17:37.474263 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463056    5609 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:17:37.474263 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463058    5609 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:17:37.474263 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463061    5609 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:17:37.474263 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463064    5609 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:17:37.474263 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463066    5609 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:17:37.474263 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463069    5609 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:17:37.474263 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463072    5609 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:17:37.474263 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463074    5609 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:17:37.474263 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463077    5609 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:17:37.474263 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463080    5609 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:17:37.474263 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463083    5609 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:17:37.474263 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463086    5609 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:17:37.474263 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463088    5609 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:17:37.474727 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463091    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:17:37.474727 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463094    5609 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:17:37.474727 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463097    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:17:37.474727 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463099    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:17:37.474727 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463102    5609 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:17:37.474727 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463104    5609 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:17:37.474727 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463107    5609 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:17:37.474727 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463110    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:17:37.474727 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463112    5609 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:17:37.474727 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463115    5609 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:17:37.474727 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463118    5609 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:17:37.474727 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463121    5609 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:17:37.474727 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463124    5609 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:17:37.474727 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463126    5609 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:17:37.474727 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463129    5609 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:17:37.474727 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463132    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:17:37.474727 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463134    5609 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:17:37.474727 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463136    5609 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:17:37.474727 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463139    5609 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:17:37.475183 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463142    5609 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:17:37.475183 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463145    5609 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:17:37.475183 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463147    5609 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:17:37.475183 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463150    5609 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:17:37.475183 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463153    5609 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:17:37.475183 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463155    5609 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:17:37.475183 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463158    5609 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:17:37.475183 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463160    5609 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:17:37.475183 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463163    5609 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:17:37.475183 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463166    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:17:37.475183 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463168    5609 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:17:37.475183 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463171    5609 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:17:37.475183 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463173    5609 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:17:37.475183 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463176    5609 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:17:37.475183 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463178    5609 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:17:37.475183 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463185    5609 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:17:37.475183 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463188    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:17:37.475183 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463191    5609 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:17:37.475183 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463194    5609 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:17:37.475183 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463197    5609 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:17:37.475765 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463199    5609 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:17:37.475765 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463202    5609 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:17:37.475765 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463204    5609 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:17:37.475765 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463207    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:17:37.475765 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463210    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:17:37.475765 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463213    5609 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:17:37.475765 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463216    5609 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:17:37.475765 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463221    5609 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:17:37.475765 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463225    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:17:37.475765 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463228    5609 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:17:37.475765 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463231    5609 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:17:37.475765 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463234    5609 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:17:37.475765 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463237    5609 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:17:37.475765 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463239    5609 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:17:37.475765 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463242    5609 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:17:37.475765 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463245    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:17:37.475765 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463247    5609 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:17:37.475765 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463250    5609 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:17:37.475765 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463252    5609 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:17:37.476307 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463256    5609 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:17:37.476307 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463261    5609 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:17:37.476307 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463264    5609 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:17:37.476307 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463267    5609 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:17:37.476307 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463270    5609 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:17:37.476307 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463273    5609 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:17:37.476307 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.463276    5609 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:17:37.476307 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.463303    5609 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 03:17:37.476307 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464821    5609 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:17:37.476307 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464837    5609 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:17:37.476307 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464841    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:17:37.476307 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464845    5609 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:17:37.476307 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464848    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:17:37.476307 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464853    5609 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:17:37.476681 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464858    5609 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:17:37.476681 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464861    5609 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:17:37.476681 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464865    5609 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:17:37.476681 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464867    5609 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:17:37.476681 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464870    5609 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:17:37.476681 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464873    5609 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:17:37.476681 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464876    5609 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:17:37.476681 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464878    5609 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:17:37.476681 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464881    5609 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:17:37.476681 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464884    5609 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:17:37.476681 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464887    5609 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:17:37.476681 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464890    5609 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:17:37.476681 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464892    5609 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:17:37.476681 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464895    5609 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:17:37.476681 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464898    5609 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:17:37.476681 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464901    5609 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:17:37.476681 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464904    5609 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:17:37.476681 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464907    5609 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:17:37.476681 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464910    5609 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:17:37.477165 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464913    5609 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:17:37.477165 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464917    5609 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:17:37.477165 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464919    5609 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:17:37.477165 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464922    5609 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:17:37.477165 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464925    5609 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:17:37.477165 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464929    5609 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:17:37.477165 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464934    5609 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:17:37.477165 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464937    5609 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:17:37.477165 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464940    5609 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:17:37.477165 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464943    5609 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:17:37.477165 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464945    5609 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:17:37.477165 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464948    5609 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:17:37.477165 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464951    5609 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:17:37.477165 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464954    5609 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:17:37.477165 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464957    5609 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:17:37.477165 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464959    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:17:37.477165 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464962    5609 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:17:37.477165 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464965    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:17:37.477165 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464967    5609 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:17:37.477165 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464970    5609 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:17:37.477670 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464973    5609 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:17:37.477670 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464976    5609 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:17:37.477670 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464978    5609 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:17:37.477670 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464981    5609 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:17:37.477670 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464983    5609 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:17:37.477670 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464987    5609 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:17:37.477670 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464990    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:17:37.477670 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464993    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:17:37.477670 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464996    5609 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:17:37.477670 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.464999    5609 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:17:37.477670 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465002    5609 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:17:37.477670 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465005    5609 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:17:37.477670 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465007    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:17:37.477670 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465010    5609 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:17:37.477670 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465013    5609 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:17:37.477670 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465015    5609 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:17:37.477670 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465018    5609 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:17:37.477670 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465020    5609 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:17:37.477670 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465023    5609 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:17:37.478164 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465026    5609 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:17:37.478164 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465028    5609 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:17:37.478164 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465031    5609 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:17:37.478164 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465033    5609 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:17:37.478164 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465036    5609 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:17:37.478164 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465039    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:17:37.478164 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465041    5609 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:17:37.478164 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465044    5609 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:17:37.478164 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465046    5609 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:17:37.478164 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465049    5609 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:17:37.478164 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465052    5609 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:17:37.478164 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465054    5609 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:17:37.478164 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465057    5609 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:17:37.478164 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465059    5609 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:17:37.478164 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465062    5609 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:17:37.478164 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465065    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:17:37.478164 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465067    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:17:37.478164 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465070    5609 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:17:37.478164 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465073    5609 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:17:37.478164 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465076    5609 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:17:37.478667 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465079    5609 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:17:37.478667 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465082    5609 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:17:37.478667 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465084    5609 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:17:37.478667 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465087    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:17:37.478667 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465089    5609 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:17:37.478667 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465092    5609 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:17:37.478667 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465094    5609 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:17:37.478667 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465097    5609 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:17:37.478667 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465099    5609 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:17:37.478667 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465102    5609 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:17:37.478667 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465104    5609 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:17:37.478667 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465107    5609 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:17:37.478667 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465109    5609 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:17:37.478667 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465112    5609 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:17:37.478667 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465114    5609 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:17:37.478667 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465117    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:17:37.478667 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465120    5609 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:17:37.478667 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465122    5609 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:17:37.478667 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465125    5609 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:17:37.478667 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465127    5609 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:17:37.479152 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465130    5609 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:17:37.479152 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465133    5609 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:17:37.479152 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.465161    5609 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 03:17:37.479152 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465202    5609 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:17:37.479152 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465208    5609 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:17:37.479152 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465212    5609 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:17:37.479152 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465216    5609 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:17:37.479152 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465218    5609 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:17:37.479152 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465221    5609 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:17:37.479152 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465224    5609 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:17:37.479152 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465227    5609 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:17:37.479152 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465231    5609 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:17:37.479152 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465234    5609 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:17:37.479152 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465237    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:17:37.479152 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465239    5609 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:17:37.479152 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465242    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:17:37.479598 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465245    5609 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:17:37.479598 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465247    5609 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:17:37.479598 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465250    5609 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:17:37.479598 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465253    5609 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:17:37.479598 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465255    5609 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:17:37.479598 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465258    5609 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:17:37.479598 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465260    5609 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:17:37.479598 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465263    5609 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:17:37.479598 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465266    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:17:37.479598 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465268    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:17:37.479598 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465271    5609 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:17:37.479598 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465273    5609 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:17:37.479598 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465276    5609 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:17:37.479598 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465279    5609 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:17:37.479598 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465282    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:17:37.479598 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465284    5609 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:17:37.479598 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465287    5609 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:17:37.479598 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465290    5609 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:17:37.479598 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465292    5609 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:17:37.479598 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465295    5609 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:17:37.480087 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465297    5609 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:17:37.480087 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465300    5609 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:17:37.480087 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465303    5609 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:17:37.480087 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465305    5609 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:17:37.480087 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465307    5609 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:17:37.480087 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465310    5609 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:17:37.480087 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465313    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:17:37.480087 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465315    5609 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:17:37.480087 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465318    5609 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:17:37.480087 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465321    5609 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:17:37.480087 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465324    5609 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:17:37.480087 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465326    5609 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:17:37.480087 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465329    5609 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:17:37.480087 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465331    5609 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:17:37.480087 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465334    5609 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:17:37.480087 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465337    5609 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:17:37.480087 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465339    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:17:37.480087 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465342    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:17:37.480087 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465344    5609 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:17:37.480634 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465347    5609 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:17:37.480634 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465350    5609 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:17:37.480634 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465352    5609 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:17:37.480634 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465355    5609 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:17:37.480634 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465358    5609 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:17:37.480634 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465360    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:17:37.480634 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465363    5609 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:17:37.480634 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465365    5609 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:17:37.480634 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465368    5609 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:17:37.480634 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465370    5609 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:17:37.480634 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465373    5609 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:17:37.480634 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465375    5609 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:17:37.480634 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465378    5609 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:17:37.480634 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465381    5609 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:17:37.480634 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465384    5609 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:17:37.480634 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465388    5609 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:17:37.480634 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465391    5609 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:17:37.480634 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465393    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:17:37.480634 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465396    5609 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:17:37.480634 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465421    5609 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:17:37.481451 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465427    5609 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:17:37.481451 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465431    5609 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:17:37.481451 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465434    5609 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:17:37.481451 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465438    5609 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:17:37.481451 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465441    5609 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:17:37.481451 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465443    5609 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:17:37.481451 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465446    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:17:37.481451 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465448    5609 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:17:37.481451 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465456    5609 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:17:37.481451 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465460    5609 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:17:37.481451 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465465    5609 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:17:37.481451 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465468    5609 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:17:37.481451 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465472    5609 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:17:37.481451 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465475    5609 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:17:37.481451 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465478    5609 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:17:37.481451 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465480    5609 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:17:37.481451 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465483    5609 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:17:37.481451 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465485    5609 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:17:37.481451 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465488    5609 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:17:37.482026 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465491    5609 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:17:37.482026 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465493    5609 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:17:37.482026 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465496    5609 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:17:37.482026 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465499    5609 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:17:37.482026 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465501    5609 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:17:37.482026 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465504    5609 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:17:37.482026 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465506    5609 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:17:37.482026 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465509    5609 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:17:37.482026 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465511    5609 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:17:37.482026 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465514    5609 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:17:37.482026 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465517    5609 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:17:37.482026 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465520    5609 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:17:37.482026 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465522    5609 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:17:37.482026 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465525    5609 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:17:37.482026 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:37.465529    5609 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:17:37.482026 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.465594    5609 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 04 03:17:37.482026 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.466610    5609 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Nov 04 03:17:37.482026 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.466662    5609 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Nov 04 03:17:37.482026 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.466836    5609 server.go:1014] "Starting client certificate rotation"
Nov 04 03:17:37.482585 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.466976    5609 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 04 03:17:37.482585 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.467026    5609 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-03 03:11:25 +0000 UTC" deadline="2028-07-11 22:54:34.423599258 +0000 UTC"
Nov 04 03:17:37.482585 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.467057    5609 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14779h36m56.95654383s"
Nov 04 03:17:37.482585 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.467288    5609 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 03:17:37.482585 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.468377    5609 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 03:17:37.482585 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.469742    5609 log.go:25] "Validated CRI v1 runtime API"
Nov 04 03:17:37.482585 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.471564    5609 log.go:25] "Validated CRI v1 image API"
Nov 04 03:17:37.482585 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.472299    5609 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 04 03:17:37.482585 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.474487    5609 fs.go:135] Filesystem UUIDs: map[008627af-74db-4dc7-968a-2fa6fb3712dc:/dev/nvme0n1p4 1a774b2d-4b73-49a9-be43-014b6c77e293:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Nov 04 03:17:37.482856 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.474508    5609 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/01166c007db55d3b820092642ea9c65c883b12c05090bf951978d4a1b1531bd8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/01166c007db55d3b820092642ea9c65c883b12c05090bf951978d4a1b1531bd8/userdata/shm major:0 minor:340 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/14e9b215aa751f29082b433860d3ce4732d3e20959d5580e3a34f58f006c3ad4/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/14e9b215aa751f29082b433860d3ce4732d3e20959d5580e3a34f58f006c3ad4/userdata/shm major:0 minor:388 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/162b5150b1fa0c5491cabb76b570878f66758b46a19ba0e60cd6b5de9f2a4708/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/162b5150b1fa0c5491cabb76b570878f66758b46a19ba0e60cd6b5de9f2a4708/userdata/shm major:0 minor:338 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/18e250ae36ea4dcd84c44211ccde315ef072c36d75a4208979550f27a5d728f2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/18e250ae36ea4dcd84c44211ccde315ef072c36d75a4208979550f27a5d728f2/userdata/shm major:0 minor:61 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2213f8bdf00825959d18e17459271b780f1b89136caac2eec3a8cd654924888b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2213f8bdf00825959d18e17459271b780f1b89136caac2eec3a8cd654924888b/userdata/shm major:0 minor:245 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/300f657b0a23c4828d40ac017087f089be7591f69de01ae10b35ad71c174dded/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/300f657b0a23c4828d40ac017087f089be7591f69de01ae10b35ad71c174dded/userdata/shm major:0 minor:88 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/32351bff9b6d4c90f5736f07fe59a8cab87fca6348451508e1e7cf33b95596b2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/32351bff9b6d4c90f5736f07fe59a8cab87fca6348451508e1e7cf33b95596b2/userdata/shm major:0 minor:247 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4354c79b250200492dc51c2675a2335f6c00a5f66b3cd19b4c0c27fe20b3cfd9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4354c79b250200492dc51c2675a2335f6c00a5f66b3cd19b4c0c27fe20b3cfd9/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/460c8b9aec9c460ded874aaca5a63be78eae1fd226e700ffc7f1587fe0828bbd/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/460c8b9aec9c460ded874aaca5a63be78eae1fd226e700ffc7f1587fe0828bbd/userdata/shm major:0 minor:77 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4ef5544e200695af7b630a18f1d5f1c548e3f5c6b5776ade65da67c07f469ee7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4ef5544e200695af7b630a18f1d5f1c548e3f5c6b5776ade65da67c07f469ee7/userdata/shm major:0 minor:235 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5e0193cee68746668d04cd45c7dbfb28ff8d5a48f555b3213f1549e560278900/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5e0193cee68746668d04cd45c7dbfb28ff8d5a48f555b3213f1549e560278900/userdata/shm major:0 minor:60 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6736a5a06164268694d3b79c2102d16b063d5d57f873ea08e2c53ffcad7f08c7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6736a5a06164268694d3b79c2102d16b063d5d57f873ea08e2c53ffcad7f08c7/userdata/shm major:0 minor:216 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8da439f9f72d07468b5f026f679257e73c6e569c195c803db1bcec906507c8b3/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8da439f9f72d07468b5f026f679257e73c6e569c195c803db1bcec906507c8b3/userdata/shm major:0 minor:242 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/90b9b99202c8eab287b223a0fbeccb03c667685bb4708c56bc6494a142c655ce/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/90b9b99202c8eab287b223a0fbeccb03c667685bb4708c56bc6494a142c655ce/userdata/shm major:0 minor:92 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9e95f53696261300915c372199df2f62ecad13630d51184d45c933312e5b9ca1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9e95f53696261300915c372199df2f62ecad13630d51184d45c933312e5b9ca1/userdata/shm major:0 minor:218 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a0039a17eb541c7e5cc28be3b894d6d1cefd9218af0ba5c89030e7eea937627a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a0039a17eb541c7e5cc28be3b894d6d1cefd9218af0ba5c89030e7eea937627a/userdata/shm major:0 minor:392 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a2c6ca82d9cc8cea0f09fc520c8c7d3bbc5ff0074d0c828f3a908af7bf5e7cac/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a2c6ca82d9cc8cea0f09fc520c8c7d3bbc5ff0074d0c828f3a908af7bf5e7cac/userdata/shm major:0 minor:394 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ab7a27f713f1fb60b4c3618a32d929cccdb24f0205e7a1673e82809112865f81/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ab7a27f713f1fb60b4c3618a32d929cccdb24f0205e7a1673e82809112865f81/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ad7ea138aa3469e914d4fef4e4b959583c82867a03c59c662cd027e8a5b163ae/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ad7ea138aa3469e914d4fef4e4b959583c82867a03c59c662cd027e8a5b163ae/userdata/shm major:0 minor:396 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b0d8f5dcca3cd28fec67127e455a70a4dd57c277871218844baaa61bc2e7b915/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b0d8f5dcca3cd28fec67127e455a70a4dd57c277871218844baaa61bc2e7b915/userdata/shm major:0 minor:344 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b89d7825ec0f0d5bdf3487d8703a80adb02388b74aaf54dffbd2b094acbf119d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b89d7825ec0f0d5bdf3487d8703a80adb02388b74aaf54dffbd2b094acbf119d/userdata/shm major:0 minor:390 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c6b9d14b9e7cc3372ca72b16275cf82ba5230ebee08be64970e3b3bc50af00e8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c6b9d14b9e7cc3372ca72b16275cf82ba5230ebee08be64970e3b3bc50af00e8/userdata/shm major:0 minor:342 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/cdb60b525560e15019a37aaf257c607d28f8bb9de40e391f8e06d0f43a2a4524/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/cdb60b525560e15019a37aaf257c607d28f8bb9de40e391f8e06d0f43a2a4524/userdata/shm major:0 minor:251 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d732058c584482205f0cda0ced5e810f97ab9a507e40b32658593463a317cfb1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d732058c584482205f0cda0ced5e810f97ab9a507e40b32658593463a317cfb1/userdata/shm major:0 minor:84 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e2233f4f39df1b4a827fd6740e336d1243793a92a49423b5e6d0087113b0f7fe/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e2233f4f39df1b4a827fd6740e336d1243793a92a49423b5e6d0087113b0f7fe/userdata/shm major:0 minor:237 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f0f6858bbc07364162f2e8e9791a42974915aaf66d257711b085ec12ad5badf8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f0f6858bbc07364162f2e8e9791a42974915aaf66d257711b085ec12ad5badf8/userdata/shm major:0 minor:203 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f1f81bac5a3c3e1fc69b0c3c653eb547e6c75e948f2c3a1c3b708b5b3139249e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f1f81bac5a3c3e1fc69b0c3c653eb547e6c75e948f2c3a1c3b708b5b3139249e/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f8de65ace441d748bf714ac3874c1d9d5bdca604c8b8c76dba6c256db3d1438d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f8de65ace441d748bf714ac3874c1d9d5bdca604c8b8c76dba6c256db3d1438d/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fcdc4285e3406fb4f6124f6289bba3ee08d73e97bbd67a8a91fa342b72c84c10/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fcdc4285e3406fb4f6124f6289bba3ee08d73e97bbd67a8a91fa342b72c84c10/userdata/shm major:0 minor:239 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ff6df41ba86f2e45d61b592fca5e1d28949c7c919df146f3ce07e1d0856c3a2a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ff6df41ba86f2e45d61b592fca5e1d28949c7c919df146f3ce07e1d0856c3a2a/userdata/shm major:0 minor:241 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/029e7b7b-265e-459a-bc09-a90b5c179993/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/029e7b7b-265e-459a-bc09-a90b5c179993/volumes/kubernetes.io~secret/agent-certs major:0 minor:48 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/04b955ef-528b-4573-b18f-0957658f79e4/volumes/kubernetes.io~projected/kube-api-access-jltvv:{mountpoint:/var/lib/kubelet/pods/04b955ef-528b-4573-b18f-0957658f79e4/volumes/kubernetes.io~projected/kube-api-access-jltvv major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/06698c23-887e-4ee5-9728-04098200c13d/volumes/kubernetes.io~projected/kube-api-access-v4sqp:{mountpoint:/var/lib/kubelet/pods/06698c23-887e-4ee5-9728-04098200c13d/volumes/kubernetes.io~projected/kube-api-access-v4sqp major:0 minor:232 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/06698c23-887e-4ee5-9728-04098200c13d/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/06698c23-887e-4ee5-9728-04098200c13d/volumes/kubernetes.io~secret/metrics-tls major:0 minor:330 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/068619f8-7fbb-4b22-ac7d-a1f144e084c0/volumes/kubernetes.io~projected/kube-api-access-mcm4d:{mountpoint:/var/lib/kubelet/pods/068619f8-7fbb-4b22-ac7d-a1f144e084c0/volumes/kubernetes.io~projected/kube-api-access-mcm4d major:0 minor:225 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/068619f8-7fbb-4b22-ac7d-a1f144e084c0/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/068619f8-7fbb-4b22-ac7d-a1f144e084c0/volumes/kubernetes.io~secret/serving-cert major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0beb091e-5c2f-4136-93a2-771f9b861411/volumes/kubernetes.io~projected/kube-api-access-m4pj6:{mountpoint:/var/lib/kubelet/pods/0beb091e-5c2f-4136-93a2-771f9b861411/volumes/kubernetes.io~projected/kube-api-access-m4pj6 major:0 minor:221 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0beb091e-5c2f-4136-93a2-771f9b861411/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/0beb091e-5c2f-4136-93a2-771f9b861411/volumes/kubernetes.io~secret/serving-cert major:0 minor:214 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/10cd06d4-ef1a-4ecf-a72c-e6d0084f3032/volumes/kubernetes.io~projected/kube-api-access-597ml:{mountpoint:/var/lib/kubelet/pods/10cd06d4-ef1a-4ecf-a72c-e6d0084f3032/volumes/kubernetes.io~projected/kube-api-access-597ml major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/21a42913-5c4e-4057-aae5-2804038f16e4/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/21a42913-5c4e-4057-aae5-2804038f16e4/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/21a42913-5c4e-4057-aae5-2804038f16e4/volumes/kubernetes.io~projected/kube-api-access-9trx7:{mountpoint:/var/lib/kubelet/pods/21a42913-5c4e-4057-aae5-2804038f16e4/volumes/kubernetes.io~projected/kube-api-access-9trx7 major:0 minor:52 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/21a42913-5c4e-4057-aae5-2804038f16e4/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/21a42913-5c4e-4057-aae5-2804038f16e4/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:49 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3b178ce9-e140-4420-81f1-627de00b90c6/volumes/kubernetes.io~projected/kube-api-access-ghplp:{mountpoint:/var/lib/kubelet/pods/3b178ce9-e140-4420-81f1-627de00b90c6/volumes/kubernetes.io~projected/kube-api-access-ghplp major:0 minor:233 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3b178ce9-e140-4420-81f1-627de00b90c6/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/3b178ce9-e140-4420-81f1-627de00b90c6/volumes/kubernetes.io~secret/serving-cert major:0 minor:212 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/45621191-5080-4ee9-a08e-f7b2ee49fe41/volumes/kubernetes.io~projected/kube-api-access-6c8b9:{mountpoint:/var/lib/kubelet/pods/45621191-5080-4ee9-a08e-f7b2ee49fe41/volumes/kubernetes.io~projected/kube-api-access-6c8b9 major:0 minor:51 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/536a0057-a438-4ad9-a4cd-b26cb7f3c645/volumes/kubernetes.io~projected/kube-api-access-7fhxt:{mountpoint:/var/lib/kubelet/pods/536a0057-a438-4ad9-a4cd-b26cb7f3c645/volumes/kubernetes.io~projected/kube-api-access-7fhxt major:0 minor:313 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/55c4af4c-188b-495b-9052-5700a1f60602/volumes/kubernetes.io~projected/kube-api-access-5h469:{mountpoint:/var/lib/kubelet/pods/55c4af4c-188b-495b-9052-5700a1f60602/volumes/kubernetes.io~projected/kube-api-access-5h469 major:0 minor:222 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/55c4af4c-188b-495b-9052-5700a1f60602/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/55c4af4c-188b-495b-9052-5700a1f60602/volumes/kubernetes.io~secret/cert major:0 minor:205 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6dedccdc-aea2-4664-8480-ca3c9359437d/volumes/kubernetes.io~projected/kube-api-access-5mbsz:{mountpoint:/var/lib/kubelet/pods/6dedccdc-aea2-4664-8480-ca3c9359437d/volumes/kubernetes.io~projected/kube-api-access-5mbsz major:0 minor:231 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6dedccdc-aea2-4664-8480-ca3c9359437d/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/6dedccdc-aea2-4664-8480-ca3c9359437d/volumes/kubernetes.io~secret/serving-cert major:0 minor:215 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6f627240-24ef-4aee-9810-51f422083ca3/volumes/kubernetes.io~projected/kube-api-access-2kl5q:{mountpoint:/var/lib/kubelet/pods/6f627240-24ef-4aee-9810-51f422083ca3/volumes/kubernetes.io~projected/kube-api-access-2kl5q major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7713890e-763d-4a51-bffb-6cf8dd12fd18/volumes/kubernetes.io~projected/kube-api-access-hp4bb:{mountpoint:/var/lib/kubelet/pods/7713890e-763d-4a51-bffb-6cf8dd12fd18/volumes/kubernetes.io~projected/kube-api-access-hp4bb major:0 minor:53 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a51f26a7-0ec8-4c56-881a-98c6d625478b/volumes/kubernetes.io~projected/kube-api-access-2ldp7:{mountpoint:/var/lib/kubelet/pods/a51f26a7-0ec8-4c56-881a-98c6d625478b/volumes/kubernetes.io~projected/kube-api-access-2ldp7 major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a683bd45-33c3-43fb-9530-292d7fea6970/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/a683bd45-33c3-43fb-9530-292d7fea6970/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:50 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a683bd45-33c3-43fb-9530-292d7fea6970/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/a683bd45-33c3-43fb-9530-292d7fea6970/volumes/kubernetes.io~empty-dir/tmp major:0 minor:47 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a683bd45-33c3-43fb-9530-292d7fea6970/volumes/kubernetes.io~projected/kube-api-access-qm4f8:{mountpoint:/var/lib/kubelet/pods/a683bd45-33c3-43fb-9530-292d7fea6970/volumes/kubernetes.io~projected/kube-api-access-qm4f8 major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/aab0bd9b-cb90-4d9b-9f46-52c6c87a34cf/volumes/kubernetes.io~projected/kube-api-access-b7cm5:{mountpoint:/var/lib/kubelet/pods/aab0bd9b-cb90-4d9b-9f46-52c6c87a34cf/volumes/kubernetes.io~projected/kube-api-access-b7cm5 major:0 minor:54 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c1b21c41-4b51-4da8-8920-c38cf49554bf/volumes/kubernetes.io~projected/kube-api-access-fsjms:{mountpoint:/var/lib/kubelet/pods/c1b21c41-4b51-4da8-8920-c38cf49554bf/volumes/kubernetes.io~projected/kube-api-access-fsjms major:0 minor:220 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c1b21c41-4b51-4da8-8920-c38cf49554bf/volumes/kubernetes.io~secret/default-certificate:{mountpoint:/var/lib/kubelet/pods/c1b21c41-4b51-4da8-8920-c38cf49554bf/volumes/kubernetes.io~secret/default-certificate major:0 minor:211 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c1b21c41-4b51-4da8-8920-c38cf49554bf/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/c1b21c41-4b51-4da8-8920-c38cf49554bf/volumes/kubernetes.io~secret/metrics-certs major:0 minor:380 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c1b21c41-4b51-4da8-8920-c38cf49554bf/volumes/kubernetes.io~secret/stats-auth:{mountpoint:/var/lib/kubelet/pods/c1b21c41-4b51-4da8-8920-c38cf49554bf/volumes/kubernetes.io~secret/stats-auth major:0 minor:209 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cc469246-2293-4301-9e46-bf9d609be964/volumes/kubernetes.io~projected/kube-api-access-cnrbg:{mountpoint:/var/lib/kubelet/pods/cc469246-2293-4301-9e46-bf9d609be964/volumes/kubernetes.io~projected/kube-api-access-cnrbg major:0 minor:227 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d10e9268-df07-428b-ae25-528d9fe075d2/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/d10e9268-df07-428b-ae25-528d9fe075d2/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:387 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d604b334-07f4-40f2-bfa9-4b26c301c84c/volumes/kubernetes.io~projected/kube-api-access-n5cdl:{mountpoint:/var/lib/kubelet/pods/d604b334-07f4-40f2-bfa9-4b26c301c84c/volumes/kubernetes.io~projected/kube-api-access-n5cdl major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d731646d-23ab-42d7-ae48-974929b3d2dc/volumes/kubernetes.io~secret/networking-console-plugin-cert:{mountpoint:/var/lib/kubelet/pods/d731646d-23ab-42d7-ae48-974929b3d2dc/volumes/kubernetes.io~secret/networking-console-plugin-cert major:0 minor:337 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e0c72601-2624-4632-9f15-c19fcc489aae/volumes/kubernetes.io~projected/kube-api-access-66hms:{mountpoint:/var/lib/kubelet/pods/e0c72601-2624-4632-9f15-c19fcc489aae/volumes/kubernetes.io~projected/kube-api-access-66hms major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e451b329-bd05-44f5-9544-8283f76df9d6/volumes/kubernetes.io~projected/kube-api-access-8g8qf:{mountpoint:/var/lib/kubelet/pods/e451b329-bd05-44f5-9544-8283f76df9d6/volumes/kubernetes.io~projected/kube-api-access-8g8qf major:0 minor:223 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e487dae4-8ac9-4acd-a8ad-a0d149909475/volumes/kubernetes.io~projected/kube-api-access-vlxxz:{mountpoint:/var/lib/kubelet/pods/e487dae4-8ac9-4acd-a8ad-a0d149909475/volumes/kubernetes.io~projected/kube-api-access-vlxxz major:0 minor:224 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e487dae4-8ac9-4acd-a8ad-a0d149909475/volumes/kubernetes.io~secret/samples-operator-tls:{mountpoint:/var/lib/kubelet/pods/e487dae4-8ac9-4acd-a8ad-a0d149909475/volumes/kubernetes.io~secret/samples-operator-tls major:0 minor:384 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e87dded5-1e1b-45cb-ab2e-ae12550dfca7/volumes/kubernetes.io~projected/kube-api-access-sbrl5:{mountpoint:/var/lib/kubelet/pods/e87dded5-1e1b-45cb-ab2e-ae12550dfca7/volumes/kubernetes.io~projected/kube-api-access-sbrl5 major:0 minor:228 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e87dded5-1e1b-45cb-ab2e-ae12550dfca7/volumes/kubernetes.io~secret/cluster-monitoring-operator-tls:{mountpoint:/var/lib/kubelet/pods/e87dded5-1e1b-45cb-ab2e-ae12550dfca7/volumes/kubernetes.io~secret/cluster-monitoring-operator-tls major:0 minor:385 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f1f61b42-3b07-4892-90a7-2efa024e435b/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/f1f61b42-3b07-4892-90a7-2efa024e435b/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:230 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f1f61b42-3b07-4892-90a7-2efa024e435b/volumes/kubernetes.io~projected/kube-api-access-kjtnj:{mountpoint:/var/lib/kubelet/pods/f1f61b42-3b07-4892-90a7-2efa024e435b/volumes/kubernetes.io~projected/kube-api-access-kjtnj major:0 minor:226 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f1f61b42-3b07-4892-90a7-2efa024e435b/volumes/kubernetes.io~projected/registry-tls:{mountpoint:/var/lib/kubelet/pods/f1f61b42-3b07-4892-90a7-2efa024e435b/volumes/kubernetes.io~projected/registry-tls major:0 minor:336 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f1f61b42-3b07-4892-90a7-2efa024e435b/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/f1f61b42-3b07-4892-90a7-2efa024e435b/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:207 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f1f61b42-3b07-4892-90a7-2efa024e435b/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/f1f61b42-3b07-4892-90a7-2efa024e435b/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:213 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fc3d3c0e-2877-4264-93a7-be3756d2dc5c/volumes/kubernetes.io~projected/kube-api-access-5krfn:{mountpoint:/var/lib/kubelet/pods/fc3d3c0e-2877-4264-93a7-be3756d2dc5c/volumes/kubernetes.io~projected/kube-api-access-5krfn major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fc3d3c0e-2877-4264-93a7-be3756d2dc5c/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/fc3d3c0e-2877-4264-93a7-be3756d2dc5c/volumes/kubernetes.io~secret/metrics-certs major:0 minor:386 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fc612506-b699-44bc-a81b-06fee4e819fe/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/fc612506-b699-44bc-a81b-06fee4e819fe/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:234 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fc612506-b699-44bc-a81b-06fee4e819fe/volumes/kubernetes.io~projected/kube-api-access-bfc98:{mountpoint:/var/lib/kubelet/pods/fc612506-b699-44bc-a81b-06fee4e819fe/volumes/kubernetes.io~projected/kube-api-access-bfc98 major:0 minor:229 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fc612506-b699-44bc-a81b-06fee4e819fe/volumes/kubernetes.io~projected/registry-tls:{mountpoint:/var/lib/kubelet/pods/fc612506-b699-44bc-a81b-06fee4e819fe/volumes/kubernetes.io~projected/registry-tls major:0 minor:335 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fc612506-b699-44bc-a81b-06fee4e819fe/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/fc612506-b699-44bc-a81b-06fee4e819fe/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:210 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fc612506-b699-44bc-a81b-06fee4e819fe/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/fc612506-b699-44bc-a81b-06fee4e819fe/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:208 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ff9f5afb-957c-4383-8fa9-698f7ada29b0/volumes/kubernetes.io~projected/kube-api-access-x424d:{mountpoint:/var/lib/kubelet/pods/ff9f5afb-957c-4383-8fa9-698f7ada29b0/volumes/kubernetes.io~projected/kube-api-access-x424d major:0 minor:57 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/5dbb3029c8b74ccfc2828d766a4bd502ffc554aad21031d74174705a5b7073d7/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/0678e795a54c9f80b80afad0ba76a24d3dce682985c615fb1c13353f1a252164/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/022bc7bcc5abbeedf534c9242dbbf608c0f4b25a16e75342dca948ac99d96924/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/68f56ee82148fb6ad7af8e19f1eadb3d5cf891e6dede96b8549b7b950051d0e0/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/4352a45792e4f7e7a792f039ea5c2710eee37b7b28ed08495224604781fed482/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/1900e34bbd8111f2d56d25006cf9b3c13da8397551981edf4d58011442375bfd/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/98d238d1f9cec0ffe36421c2966cbab7273b04aa0d5f53dbacb7009ebf53490f/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/d6a308f1727fa42f3e4e8d83c0e5ed43fb0f6a581fc1a543c150d36787006c91/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/19e6b7e93da313f2e1e142b4d1029f4b5a35cc6ae8784900c63d8f0d7f7e70c5/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-137:{mountpoint:/var/lib/containers/storage/overlay/278052418c3331909ed2673842bb93023000285e516176abd6be9ad767fd2089/merged major:0 minor:137 fsType:overlay blockSize:0} overlay_0-139:{mountpoint:/var/lib/containers/storage/overlay/fbc3755e53f22617bea662495e3a003e6a807a50d14c03ce57d0372990c2b8d9/merged major:0 minor:139 fsType:overlay blockSize:0} overlay_0-147:{mountpoint:/var/lib/containers/storage/overlay/a000019b1c5e9fb2658f69bf661654aec795689f2bb770962f7c1972fa099ffd/merged major:0 minor:147 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/faeecfd0a0691106c51f1b52c7dda386ed80a9d462d92d2e5a35db7810aa8ce4/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/66a80c821d2cf8ca6e2d1134523572af89d87a2a4b844d87a18193c7fbf27efb/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/21e6ff4a521ed81fb0ce6afb2aa69689147c57328fc5cdc845dd9d2a977668ed/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/de45e4268d111920fb862c405b2c832923ce09e3f8d670e5c2bd252627f1873a/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/5723b6acc64d07e2682220c60dedd7ecb9c5e59a69d6ba66942c994c89e2b869/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/20293d2b2aa7e6bd655a4be981ade285fb140bbfbf83819be3e330120f83ab98/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/81be4cb3506c12ae81b1a92b2729ff94a58ccbe0d9d409dfb04c89d256ae0b35/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/003735c6797efee88d200eb85d72b313ec47c9d058aabed6085c1a58f57f8039/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/4e818f9c61fc38da95f25c2d91e23f7c938971312b6a8ea3e2feb88e48f1bac9/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-249:{mountpoint:/var/lib/containers/storage/overlay/ed3b46492a7867e6e56c2da216151e50984b232784a57d2b7a12cc5a233f795a/merged major:0 minor:249 fsType:overlay blockSize:0} overlay_0-253:{mountpoint:/var/lib/containers/storage/overlay/7a87d44d616b66a60a15291174e0104a4c709a133b8ab657080cb23cde8d56ee/merged major:0 minor:253 fsType:overlay blockSize:0} overlay_0-255:{mountpoint:/var/lib/containers/storage/overlay/357cc5d0fd470c59ac9ab4c9af971de505e52606ed8fffbfbfd24ac13a952481/merged major:0 minor:255 fsType:overlay blockSize:0} overlay_0-257:{mountpoint:/var/lib/containers/storage/overlay/72b871fc5463442678d981cec696e4ba4f40b25d81ee1beea5a91771f0b22fe5/merged major:0 minor:257 fsType:overlay blockSize:0} overlay_0-259:{mountpoint:/var/lib/containers/storage/overlay/cb0071ca9f37148d0e09c750967d70e7a7b133d094003c6dcc11b3149ec1df9e/merged major:0 minor:259 fsType:overlay blockSize:0} overlay_0-261:{mountpoint:/var/lib/containers/storage/overlay/79b03283aa8aa755fc22d3bd1d05bfee650ad63db8143164559d317740c6858a/merged major:0 minor:261 fsType:overlay blockSize:0} overlay_0-263:{mountpoint:/var/lib/containers/storage/overlay/6aff62efae18ade6d945477314a39e36831738cb7923ff5c56b53fbfc3cd23f1/merged major:0 minor:263 fsType:overlay blockSize:0} overlay_0-264:{mountpoint:/var/lib/containers/storage/overlay/c7f97bbf4d5b7609858b15f0949a7e59bdbcb9762f3dfe28ab75ef7eb8f89edb/merged major:0 minor:264 fsType:overlay blockSize:0} overlay_0-270:{mountpoint:/var/lib/containers/storage/overlay/6365f0d27e9969236513247c551decfc58023fd028c8765b5a8acb47ff2a01cc/merged major:0 minor:270 fsType:overlay blockSize:0} overlay_0-273:{mountpoint:/var/lib/containers/storage/overlay/7e82a49467178aea468e2f0e842fbeb4c341d36b79cb0406fda319d85609ba4e/merged major:0 minor:273 fsType:overlay blockSize:0} overlay_0-274:{mountpoint:/var/lib/containers/storage/overlay/b598a60e16e65652468c8a1e5624db9108c798bcdca9e046c376ce9618741753/merged major:0 minor:274 fsType:overlay blockSize:0} overlay_0-285:{mountpoint:/var/lib/containers/storage/overlay/4a65baefe1b81b59c6a729ca2e70d8a57e44e180a159d15aeeaa30927c79c122/merged major:0 minor:285 fsType:overlay blockSize:0} overlay_0-287:{mountpoint:/var/lib/containers/storage/overlay/b7fb5476a64c457921020a1b7370c9af9e3da71b4d1774b6bf3474742d4e483d/merged major:0 minor:287 fsType:overlay blockSize:0} overlay_0-289:{mountpoint:/var/lib/containers/storage/overlay/ff5996f7aaf57512db2792fbf2fb6682418ff0039656e3744b32d073d6828346/merged major:0 minor:289 fsType:overlay blockSize:0} overlay_0-291:{mountpoint:/var/lib/containers/storage/overlay/b3b1b0da46cf0f067cf7a69bd8b9647d18c648a30c7bf46a2be322b7c2c8feb1/merged major:0 minor:291 fsType:overlay blockSize:0} overlay_0-293:{mountpoint:/var/lib/containers/storage/overlay/7fa64488137ccd2f95d75e0e342640c65e36ed1f60464eca9689c8a65fdfacc4/merged major:0 minor:293 fsType:overlay blockSize:0} overlay_0-299:{mountpoint:/var/lib/containers/storage/overlay/5ebc3cc023a78da176ae882c70a3afb61bf4416cdfbc0d1476980b698ef731cc/merged major:0 minor:299 fsType:overlay blockSize:0} overlay_0-305:{mountpoint:/var/lib/containers/storage/overlay/1880807e6ed08348df1a3fb515ee2e454c889a62a46588c5dc7aa602a237b803/merged major:0 minor:305 fsType:overlay blockSize:0} overlay_0-311:{mountpoint:/var/lib/containers/storage/overlay/13807132f14b5d72098b33e6e8bc57eb0d18e3b4da31416905414dbea41db00f/merged major:0 minor:311 fsType:overlay blockSize:0} overlay_0-318:{mountpoint:/var/lib/containers/storage/overlay/de57c7a20b56e3eb811da3d2697328b6f46084e03b1938617b6fae01c827859c/merged major:0 minor:318 fsType:overlay blockSize:0} overlay_0-324:{mountpoint:/var/lib/containers/storage/overlay/658e976eb2202754075e0ada9fdf1550ea66abe4c3937da5b4ac9aa9975acd33/merged major:0 minor:324 fsType:overlay blockSize:0} overlay_0-346:{mountpoint:/var/lib/containers/storage/overlay/8d362e166c7e4a398b9750149c6dab3759dab1d274a8425d4b6a4e052fe7bfdf/merged major:0 minor:346 fsType:overlay blockSize:0} overlay_0-348:{mountpoint:/var/lib/containers/storage/overlay/0beffe69455fed012a37d2712e9e553e4985e286afc56d621b5d04838369f74f/merged major:0 minor:348 fsType:overlay blockSize:0} overlay_0-350:{mountpoint:/var/lib/containers/storage/overlay/e3da1463d0c66e7433c4556f52a51dc8d70b0852b5cd8ab36574405cd3449469/merged major:0 minor:350 fsType:overlay blockSize:0} overlay_0-352:{mountpoint:/var/lib/containers/storage/overlay/f461b518ebc2b5b699f55c1d459dbe4092c250f1fb4077e1a74fd1156e034e3a/merged major:0 minor:352 fsType:overlay blockSize:0} overlay_0-354:{mountpoint:/var/lib/containers/storage/overlay/f2507e3403388f79b6f69accac773b2febd9ddf6c91561ba01f5702f748dec3f/merged major:0 minor:354 fsType:overlay blockSize:0} overlay_0-360:{mountpoint:/var/lib/containers/storage/overlay/b14eef83f783cc85e338aec62b61aba0267655054651f922ec08a61202ceea49/merged major:0 minor:360 fsType:overlay blockSize:0} overlay_0-362:{mountpoint:/var/lib/containers/storage/overlay/c3529b926de7f47cd46e9162f5e8a7a298456f78e886911572daf8095009e1d0/merged major:0 minor:362 fsType:overlay blockSize:0} overlay_0-368:{mountpoint:/var/lib/containers/storage/overlay/b2b07c239184424ad9d25831f887036226a18683a9ada537ffe5c99cff120b3d/merged major:0 minor:368 fsType:overlay blockSize:0} overlay_0-374:{mountpoint:/var/lib/containers/storage/overlay/5b46846f143cda75c10f2ec5ecbafe6856366407a57d7c4217526905513f9baf/merged major:0 minor:374 fsType:overlay blockSize:0} overlay_0-398:{mountpoint:/var/lib/containers/storage/overlay/0efcf81fe8a93d169ef81f46fe85ecfc52b699fbf3ce27c4e72e98f1385c0034/merged major:0 minor:398 fsType:overlay blockSize:0} overlay_0-400:{mountpoint:/var/lib/containers/storage/overlay/1c14694a2823ba248262c994ebcfc0cba718e52d0749afe8f825c9ee74f85d0a/merged major:0 minor:400 fsType:overlay blockSize:0} overlay_0-402:{mountpoint:/var/lib/containers/storage/overlay/6933c713eec4ab3654b738c72b0779eff913d31a9515a24dbbfa7a707a97e5af/merged major:0 minor:402 fsType:overlay blockSize:0} overlay_0-403:{mountpoint:/var/lib/containers/storage/overlay/a14485b78ef56ae525ee8886cc49d18909e79b7e638d0f8277acbbe46ddab17c/merged major:0 minor:403 fsType:overlay blockSize:0} overlay_0-410:{mountpoint:/var/lib/containers/storage/overlay/c589a052ea516cba411a61ec5c4aecdcff6e9970d4709c33b6c57b71193e54d5/merged major:0 minor:410 fsType:overlay blockSize:0} overlay_0-416:{mountpoint:/var/lib/containers/storage/overlay/b03092aa64cda03bb8f6fd106896d90e9de47cbe540645f3db587e3611813a5b/merged major:0 minor:416 fsType:overlay blockSize:0} overlay_0-418:{mountpoint:/var/lib/containers/storage/overlay/644e0597d3757747ecf7afc74ec3e1d6f3527958d5d312e307f5a211cae069b6/merged major:0 minor:418 fsType:overlay blockSize:0} overlay_0-420:{mountpoint:/var/lib/containers/storage/overlay/d8c59680bc238cab97d8f5124132154e08d6c2a7c9a3d4dfc6adcd121e1c4f15/merged major:0 minor:420 fsType:overlay blockSize:0} overlay_0-426:{mountpoint:/var/lib/containers/storage/overlay/c548e9fa25fe4308a748656df6e7302b5369e1c3e0d4597c42181daa6a15653b/merged major:0 minor:426 fsType:overlay blockSize:0} overlay_0-431:{mountpoint:/var/lib/containers/storage/overlay/e4b4ef13e49379ad48fee2639e188bb4d5368317f46e1b3c9f1bba7ecc6ec61c/merged major:0 minor:431 fsType:overlay blockSize:0} overlay_0-435:{mountpoint:/var/lib/containers/storage/overlay/79fb98f0bb2c0b8212ea181f81f0424603590b04204a9f0bde98b15d9951e21d/merged major:0 minor:435 fsType:overlay blockSize:0} overlay_0-438:{mountpoint:/var/lib/containers/storage/overlay/7da1d09014202fe2390f628018c3e563e2496b947b993d59fedf5327e944b97e/merged major:0 minor:438 fsType:overlay blockSize:0} overlay_0-64:{mountpoint:/var/lib/containers/storage/overlay/bc97b7444056a3ba4282287492db36936735ba1a55967aa7c1799b823aedffc6/merged major:0 minor:64 fsType:overlay blockSize:0} overlay_0-66:{mountpoint:/var/lib/containers/storage/overlay/dd6ba872536667a4f4884621b309407996a4d00e17659787518a6059fc7a13c2/merged major:0 minor:66 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/a188dbe5e472b7a5e267c09e7561e641b69cf06d56375d961f898cbd81df34a9/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/c6f2af83c8ee4cc6ba05846c608e234f34a0b38c6807f9a948e8f07681f476bc/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-79:{mountpoint:/var/lib/containers/storage/overlay/f0bfd7e37df8c27dd883f165971cc3120eecc32947eb713ac8fd73c5201beb3e/merged major:0 minor:79 fsType:overlay blockSize:0} overlay_0-82:{mountpoint:/var/lib/containers/storage/overlay/d5a0011372852810cd615826b8aa301ef4ac9bcc62a4b2099623f89a22d9d590/merged major:0 minor:82 fsType:overlay blockSize:0} overlay_0-86:{mountpoint:/var/lib/containers/storage/overlay/015c94484dbc8e2ed6c3a2f66fb1c9caa88b89af75bc5ec03c206541c206c911/merged major:0 minor:86 fsType:overlay blockSize:0} overlay_0-90:{mountpoint:/var/lib/containers/storage/overlay/3c849ed6b430489a2a29e6e4b077a9824002674ab4adc18f2adfcf8206d1feed/merged major:0 minor:90 fsType:overlay blockSize:0} overlay_0-94:{mountpoint:/var/lib/containers/storage/overlay/cf6c5a3d82a7225f63a6c21e11aa48f1f367e99c1493cb2c691f273bc455e3dd/merged major:0 minor:94 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/029d2818927322618e0de9af42c9707e4def819957a1cd3953cdaf0f771e0514/merged major:0 minor:96 fsType:overlay blockSize:0}]
Nov 04 03:17:37.483719 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.483316    5609 manager.go:217] Machine: {Timestamp:2026-11-04 03:17:37.482829555 +0000 UTC m=+0.059482460 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3104276 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ed8b0b8838639e66c5736906615ef SystemUUID:ec2ed8b0-b883-8639-e66c-5736906615ef BootID:1d9cfbae-33ed-4515-b116-6962f7c263ff Filesystems:[{Device:overlay_0-259 DeviceMajor:0 DeviceMinor:259 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-410 DeviceMajor:0 DeviceMinor:410 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f1f61b42-3b07-4892-90a7-2efa024e435b/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:213 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6736a5a06164268694d3b79c2102d16b063d5d57f873ea08e2c53ffcad7f08c7/userdata/shm DeviceMajor:0 DeviceMinor:216 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9e95f53696261300915c372199df2f62ecad13630d51184d45c933312e5b9ca1/userdata/shm DeviceMajor:0 DeviceMinor:218 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/14e9b215aa751f29082b433860d3ce4732d3e20959d5580e3a34f58f006c3ad4/userdata/shm DeviceMajor:0 DeviceMinor:388 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5e0193cee68746668d04cd45c7dbfb28ff8d5a48f555b3213f1549e560278900/userdata/shm DeviceMajor:0 DeviceMinor:60 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/a51f26a7-0ec8-4c56-881a-98c6d625478b/volumes/kubernetes.io~projected/kube-api-access-2ldp7 DeviceMajor:0 DeviceMinor:202 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e0c72601-2624-4632-9f15-c19fcc489aae/volumes/kubernetes.io~projected/kube-api-access-66hms DeviceMajor:0 DeviceMinor:197 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-289 DeviceMajor:0 DeviceMinor:289 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fc612506-b699-44bc-a81b-06fee4e819fe/volumes/kubernetes.io~projected/registry-tls DeviceMajor:0 DeviceMinor:335 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a2c6ca82d9cc8cea0f09fc520c8c7d3bbc5ff0074d0c828f3a908af7bf5e7cac/userdata/shm DeviceMajor:0 DeviceMinor:394 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/460c8b9aec9c460ded874aaca5a63be78eae1fd226e700ffc7f1587fe0828bbd/userdata/shm DeviceMajor:0 DeviceMinor:77 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-86 DeviceMajor:0 DeviceMinor:86 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-137 DeviceMajor:0 DeviceMinor:137 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-139 DeviceMajor:0 DeviceMinor:139 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7713890e-763d-4a51-bffb-6cf8dd12fd18/volumes/kubernetes.io~projected/kube-api-access-hp4bb DeviceMajor:0 DeviceMinor:53 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/6f627240-24ef-4aee-9810-51f422083ca3/volumes/kubernetes.io~projected/kube-api-access-2kl5q DeviceMajor:0 DeviceMinor:58 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-90 DeviceMajor:0 DeviceMinor:90 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6dedccdc-aea2-4664-8480-ca3c9359437d/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:215 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/fc612506-b699-44bc-a81b-06fee4e819fe/volumes/kubernetes.io~projected/kube-api-access-bfc98 DeviceMajor:0 DeviceMinor:229 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ab7a27f713f1fb60b4c3618a32d929cccdb24f0205e7a1673e82809112865f81/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/f1f61b42-3b07-4892-90a7-2efa024e435b/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:230 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b0d8f5dcca3cd28fec67127e455a70a4dd57c277871218844baaa61bc2e7b915/userdata/shm DeviceMajor:0 DeviceMinor:344 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-350 DeviceMajor:0 DeviceMinor:350 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/21a42913-5c4e-4057-aae5-2804038f16e4/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:49 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-82 DeviceMajor:0 DeviceMinor:82 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f0f6858bbc07364162f2e8e9791a42974915aaf66d257711b085ec12ad5badf8/userdata/shm DeviceMajor:0 DeviceMinor:203 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/068619f8-7fbb-4b22-ac7d-a1f144e084c0/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:206 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0beb091e-5c2f-4136-93a2-771f9b861411/volumes/kubernetes.io~projected/kube-api-access-m4pj6 DeviceMajor:0 DeviceMinor:221 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-291 DeviceMajor:0 DeviceMinor:291 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-299 DeviceMajor:0 DeviceMinor:299 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/55c4af4c-188b-495b-9052-5700a1f60602/volumes/kubernetes.io~projected/kube-api-access-5h469 DeviceMajor:0 DeviceMinor:222 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b89d7825ec0f0d5bdf3487d8703a80adb02388b74aaf54dffbd2b094acbf119d/userdata/shm DeviceMajor:0 DeviceMinor:390 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4354c79b250200492dc51c2675a2335f6c00a5f66b3cd19b4c0c27fe20b3cfd9/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f1f61b42-3b07-4892-90a7-2efa024e435b/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:207 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-263 DeviceMajor:0 DeviceMinor:263 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ad7ea138aa3469e914d4fef4e4b959583c82867a03c59c662cd027e8a5b163ae/userdata/shm DeviceMajor:0 DeviceMinor:396 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/45621191-5080-4ee9-a08e-f7b2ee49fe41/volumes/kubernetes.io~projected/kube-api-access-6c8b9 DeviceMajor:0 DeviceMinor:51 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-66 DeviceMajor:0 DeviceMinor:66 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-94 DeviceMajor:0 DeviceMinor:94 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fc612506-b699-44bc-a81b-06fee4e819fe/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:210 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e2233f4f39df1b4a827fd6740e336d1243793a92a49423b5e6d0087113b0f7fe/userdata/shm DeviceMajor:0 DeviceMinor:237 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8da439f9f72d07468b5f026f679257e73c6e569c195c803db1bcec906507c8b3/userdata/shm DeviceMajor:0 DeviceMinor:242 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e487dae4-8ac9-4acd-a8ad-a0d149909475/volumes/kubernetes.io~secret/samples-operator-tls DeviceMajor:0 DeviceMinor:384 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-255 DeviceMajor:0 DeviceMinor:255 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-400 DeviceMajor:0 DeviceMinor:400 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d732058c584482205f0cda0ced5e810f97ab9a507e40b32658593463a317cfb1/userdata/shm DeviceMajor:0 DeviceMinor:84 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/55c4af4c-188b-495b-9052-5700a1f60602/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:205 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/068619f8-7fbb-4b22-ac7d-a1f144e084c0/volumes/kubernetes.io~projected/kube-api-access-mcm4d DeviceMajor:0 DeviceMinor:225 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e87dded5-1e1b-45cb-ab2e-ae12550dfca7/volumes/kubernetes.io~projected/kube-api-access-sbrl5 DeviceMajor:0 DeviceMinor:228 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/6dedccdc-aea2-4664-8480-ca3c9359437d/volumes/kubernetes.io~projected/kube-api-access-5mbsz DeviceMajor:0 DeviceMinor:231 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fcdc4285e3406fb4f6124f6289bba3ee08d73e97bbd67a8a91fa342b72c84c10/userdata/shm DeviceMajor:0 DeviceMinor:239 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-287 DeviceMajor:0 DeviceMinor:287 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fc3d3c0e-2877-4264-93a7-be3756d2dc5c/volumes/kubernetes.io~projected/kube-api-access-5krfn DeviceMajor:0 DeviceMinor:59 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-264 DeviceMajor:0 DeviceMinor:264 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c6b9d14b9e7cc3372ca72b16275cf82ba5230ebee08be64970e3b3bc50af00e8/userdata/shm DeviceMajor:0 DeviceMinor:342 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-403 DeviceMajor:0 DeviceMinor:403 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-426 DeviceMajor:0 DeviceMinor:426 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/90b9b99202c8eab287b223a0fbeccb03c667685bb4708c56bc6494a142c655ce/userdata/shm DeviceMajor:0 DeviceMinor:92 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/536a0057-a438-4ad9-a4cd-b26cb7f3c645/volumes/kubernetes.io~projected/kube-api-access-7fhxt DeviceMajor:0 DeviceMinor:313 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-418 DeviceMajor:0 DeviceMinor:418 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-431 DeviceMajor:0 DeviceMinor:431 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d604b334-07f4-40f2-bfa9-4b26c301c84c/volumes/kubernetes.io~projected/kube-api-access-n5cdl DeviceMajor:0 DeviceMinor:55 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/21a42913-5c4e-4057-aae5-2804038f16e4/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/04b955ef-528b-4573-b18f-0957658f79e4/volumes/kubernetes.io~projected/kube-api-access-jltvv DeviceMajor:0 DeviceMinor:45 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/fc612506-b699-44bc-a81b-06fee4e819fe/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:208 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/162b5150b1fa0c5491cabb76b570878f66758b46a19ba0e60cd6b5de9f2a4708/userdata/shm DeviceMajor:0 DeviceMinor:338 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-346 DeviceMajor:0 DeviceMinor:346 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-352 DeviceMajor:0 DeviceMinor:352 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d10e9268-df07-428b-ae25-528d9fe075d2/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:387 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c1b21c41-4b51-4da8-8920-c38cf49554bf/volumes/kubernetes.io~projected/kube-api-access-fsjms DeviceMajor:0 DeviceMinor:220 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e487dae4-8ac9-4acd-a8ad-a0d149909475/volumes/kubernetes.io~projected/kube-api-access-vlxxz DeviceMajor:0 DeviceMinor:224 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/cc469246-2293-4301-9e46-bf9d609be964/volumes/kubernetes.io~projected/kube-api-access-cnrbg DeviceMajor:0 DeviceMinor:227 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/fc612506-b699-44bc-a81b-06fee4e819fe/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:234 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-273 DeviceMajor:0 DeviceMinor:273 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-324 DeviceMajor:0 DeviceMinor:324 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-420 DeviceMajor:0 DeviceMinor:420 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a683bd45-33c3-43fb-9530-292d7fea6970/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:47 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-261 DeviceMajor:0 DeviceMinor:261 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-438 DeviceMajor:0 DeviceMinor:438 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/10cd06d4-ef1a-4ecf-a72c-e6d0084f3032/volumes/kubernetes.io~projected/kube-api-access-597ml DeviceMajor:0 DeviceMinor:201 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/fc3d3c0e-2877-4264-93a7-be3756d2dc5c/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:386 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/ff9f5afb-957c-4383-8fa9-698f7ada29b0/volumes/kubernetes.io~projected/kube-api-access-x424d DeviceMajor:0 DeviceMinor:57 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/18e250ae36ea4dcd84c44211ccde315ef072c36d75a4208979550f27a5d728f2/userdata/shm DeviceMajor:0 DeviceMinor:61 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-79 DeviceMajor:0 DeviceMinor:79 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-274 DeviceMajor:0 DeviceMinor:274 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/aab0bd9b-cb90-4d9b-9f46-52c6c87a34cf/volumes/kubernetes.io~projected/kube-api-access-b7cm5 DeviceMajor:0 DeviceMinor:54 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-368 DeviceMajor:0 DeviceMinor:368 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a683bd45-33c3-43fb-9530-292d7fea6970/volumes/kubernetes.io~projected/kube-api-access-qm4f8 DeviceMajor:0 DeviceMinor:56 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-147 DeviceMajor:0 DeviceMinor:147 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4ef5544e200695af7b630a18f1d5f1c548e3f5c6b5776ade65da67c07f469ee7/userdata/shm DeviceMajor:0 DeviceMinor:235 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2213f8bdf00825959d18e17459271b780f1b89136caac2eec3a8cd654924888b/userdata/shm DeviceMajor:0 DeviceMinor:245 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/cdb60b525560e15019a37aaf257c607d28f8bb9de40e391f8e06d0f43a2a4524/userdata/shm DeviceMajor:0 DeviceMinor:251 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-305 DeviceMajor:0 DeviceMinor:305 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/06698c23-887e-4ee5-9728-04098200c13d/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:330 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-270 DeviceMajor:0 DeviceMinor:270 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-354 DeviceMajor:0 DeviceMinor:354 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-374 DeviceMajor:0 DeviceMinor:374 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c1b21c41-4b51-4da8-8920-c38cf49554bf/volumes/kubernetes.io~secret/default-certificate DeviceMajor:0 DeviceMinor:211 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/f1f61b42-3b07-4892-90a7-2efa024e435b/volumes/kubernetes.io~projected/kube-api-access-kjtnj DeviceMajor:0 DeviceMinor:226 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-253 DeviceMajor:0 DeviceMinor:253 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-318 DeviceMajor:0 DeviceMinor:318 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/01166c007db55d3b820092642ea9c65c883b12c05090bf951978d4a1b1531bd8/userdata/shm DeviceMajor:0 DeviceMinor:340 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-348 DeviceMajor:0 DeviceMinor:348 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a683bd45-33c3-43fb-9530-292d7fea6970/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:50 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f8de65ace441d748bf714ac3874c1d9d5bdca604c8b8c76dba6c256db3d1438d/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/3b178ce9-e140-4420-81f1-627de00b90c6/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:212 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-293 DeviceMajor:0 DeviceMinor:293 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d731646d-23ab-42d7-ae48-974929b3d2dc/volumes/kubernetes.io~secret/networking-console-plugin-cert DeviceMajor:0 DeviceMinor:337 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-64 DeviceMajor:0 DeviceMinor:64 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-311 DeviceMajor:0 DeviceMinor:311 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a0039a17eb541c7e5cc28be3b894d6d1cefd9218af0ba5c89030e7eea937627a/userdata/shm DeviceMajor:0 DeviceMinor:392 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f1f81bac5a3c3e1fc69b0c3c653eb547e6c75e948f2c3a1c3b708b5b3139249e/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c1b21c41-4b51-4da8-8920-c38cf49554bf/volumes/kubernetes.io~secret/stats-auth DeviceMajor:0 DeviceMinor:209 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0beb091e-5c2f-4136-93a2-771f9b861411/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:214 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/06698c23-887e-4ee5-9728-04098200c13d/volumes/kubernetes.io~projected/kube-api-access-v4sqp DeviceMajor:0 DeviceMinor:232 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/3b178ce9-e140-4420-81f1-627de00b90c6/volumes/kubernetes.io~projected/kube-api-access-ghplp DeviceMajor:0 DeviceMinor:233 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/32351bff9b6d4c90f5736f07fe59a8cab87fca6348451508e1e7cf33b95596b2/userdata/shm DeviceMajor:0 DeviceMinor:247 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-285 DeviceMajor:0 DeviceMinor:285 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-362 DeviceMajor:0 DeviceMinor:362 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/var/lib/kubelet/pods/029e7b7b-265e-459a-bc09-a90b5c179993/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:48 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/300f657b0a23c4828d40ac017087f089be7591f69de01ae10b35ad71c174dded/userdata/shm DeviceMajor:0 DeviceMinor:88 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e451b329-bd05-44f5-9544-8283f76df9d6/volumes/kubernetes.io~projected/kube-api-access-8g8qf DeviceMajor:0 DeviceMinor:223 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-257 DeviceMajor:0 DeviceMinor:257 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-360 DeviceMajor:0 DeviceMinor:360 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e87dded5-1e1b-45cb-ab2e-ae12550dfca7/volumes/kubernetes.io~secret/cluster-monitoring-operator-tls DeviceMajor:0 DeviceMinor:385 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/21a42913-5c4e-4057-aae5-2804038f16e4/volumes/kubernetes.io~projected/kube-api-access-9trx7 DeviceMajor:0 DeviceMinor:52 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-249 DeviceMajor:0 DeviceMinor:249 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f1f61b42-3b07-4892-90a7-2efa024e435b/volumes/kubernetes.io~projected/registry-tls DeviceMajor:0 DeviceMinor:336 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ff6df41ba86f2e45d61b592fca5e1d28949c7c919df146f3ce07e1d0856c3a2a/userdata/shm DeviceMajor:0 DeviceMinor:241 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/c1b21c41-4b51-4da8-8920-c38cf49554bf/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:380 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-398 DeviceMajor:0 DeviceMinor:398 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-402 DeviceMajor:0 DeviceMinor:402 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-416 DeviceMajor:0 DeviceMinor:416 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-435 DeviceMajor:0 DeviceMinor:435 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:01166c007db55d3 MacAddress:c6:33:e8:ff:06:0b Speed:10000 Mtu:8901} {Name:14e9b215aa751f2 MacAddress:42:91:50:df:5f:9e Speed:10000 Mtu:8901} {Name:162b5150b1fa0c5 MacAddress:96:fa:f0:f7:76:2a Speed:10000 Mtu:8901} {Name:2213f8bdf008259 MacAddress:a6:45:21:45:38:72 Speed:10000 Mtu:8901} {Name:32351bff9b6d4c9 MacAddress:d2:5b:bb:b8:31:5d Speed:10000 Mtu:8901} {Name:4ef5544e200695a MacAddress:86:7d:af:e9:14:ec Speed:10000 Mtu:8901} {Name:6736a5a06164268 MacAddress:ae:ed:d6:e6:09:c1 Speed:10000 Mtu:8901} {Name:8da439f9f72d074 MacAddress:ea:d0:20:30:87:33 Speed:10000 Mtu:8901} {Name:9e95f5369626130 MacAddress:7a:7f:26:f0:26:16 Speed:10000 Mtu:8901} {Name:a0039a17eb541c7 MacAddress:de:e1:5b:3e:9b:4b Speed:10000 Mtu:8901} {Name:a2c6ca82d9cc8ce MacAddress:8a:65:fb:c7:94:4b Speed:10000 Mtu:8901} {Name:ad7ea138aa3469e MacAddress:0a:a4:73:0d:34:76 Speed:10000 Mtu:8901} {Name:b0d8f5dcca3cd28 MacAddress:fa:d0:e0:b0:22:8b Speed:10000 Mtu:8901} {Name:b89d7825ec0f0d5 MacAddress:da:ae:8f:b6:af:20 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:06:ff:c7:79:1b:c3 Speed:0 Mtu:9001} {Name:br-int MacAddress:a6:3d:31:31:45:0a Speed:0 Mtu:8901} {Name:c6b9d14b9e7cc33 MacAddress:12:fe:48:a0:74:ca Speed:10000 Mtu:8901} {Name:cdb60b525560e15 MacAddress:66:35:36:8f:df:a7 Speed:10000 Mtu:8901} {Name:e2233f4f39df1b4 MacAddress:7a:55:ea:eb:c6:57 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:06:ff:c7:79:1b:c3 Speed:0 Mtu:9001} {Name:f0f6858bbc07364 MacAddress:de:6b:62:cd:c5:ed Speed:10000 Mtu:8901} {Name:fcdc4285e3406fb MacAddress:42:a4:24:ef:97:e4 Speed:10000 Mtu:8901} {Name:ff6df41ba86f2e4 MacAddress:4e:5c:3d:7a:79:82 Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:aa:f9:ad:e3:10:52 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:84:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:ea:52:c8:85:ad:0f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 04 03:17:37.483931 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.483724    5609 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 04 03:17:37.483931 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.483774    5609 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 04 03:17:37.484108 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.484083    5609 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 04 03:17:37.484238 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.484110    5609 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-50.us-west-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 04 03:17:37.484285 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.484240    5609 topology_manager.go:143] "Creating topology manager with none policy"
Nov 04 03:17:37.484285 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.484248    5609 container_manager_linux.go:311] "Creating device plugin manager"
Nov 04 03:17:37.484285 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.484259    5609 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 03:17:37.484285 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.484279    5609 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 03:17:37.484388 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.484382    5609 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 04 03:17:37.484586 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.484576    5609 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 04 03:17:37.484697 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.484688    5609 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 04 03:17:37.484731 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.484723    5609 kubelet.go:493] "Attempting to sync node with API server"
Nov 04 03:17:37.484759 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.484738    5609 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 04 03:17:37.484759 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.484747    5609 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 04 03:17:37.484759 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.484756    5609 kubelet.go:405] "Adding apiserver pod source"
Nov 04 03:17:37.484873 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.484764    5609 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 04 03:17:37.485464 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.485444    5609 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 04 03:17:37.485464 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.485457    5609 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 03:17:37.485908 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.485889    5609 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 04 03:17:37.488435 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.488416    5609 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 04 03:17:37.488435 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.488431    5609 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 03:17:37.488564 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.488525    5609 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Nov 04 03:17:37.488722 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.488711    5609 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 04 03:17:37.488753 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.488740    5609 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 04 03:17:37.488844 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.488833    5609 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 04 03:17:37.488877 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.488848    5609 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 04 03:17:37.488877 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.488855    5609 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 04 03:17:37.488877 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.488864    5609 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 04 03:17:37.488877 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.488872    5609 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 04 03:17:37.488877 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.488878    5609 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 04 03:17:37.489053 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.488884    5609 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 04 03:17:37.489053 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.488891    5609 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 04 03:17:37.489053 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.488897    5609 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 04 03:17:37.489053 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.488903    5609 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 04 03:17:37.489053 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.488911    5609 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 04 03:17:37.489053 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.488921    5609 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 04 03:17:37.489053 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.488947    5609 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 04 03:17:37.489053 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.488953    5609 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 04 03:17:37.492489 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.492476    5609 server.go:1263] "Started kubelet"
Nov 04 03:17:37.492585 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.492557    5609 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 04 03:17:37.493797 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.492641    5609 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 04 03:17:37.493797 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.492895    5609 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 04 03:17:37.493797 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.493317    5609 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 04 03:17:37.493108 ip-10-0-128-50 systemd[1]: Started Kubernetes Kubelet.
Nov 04 03:17:37.494267 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.494183    5609 server.go:317] "Adding debug handlers to kubelet server"
Nov 04 03:17:37.500504 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.499057    5609 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 04 03:17:37.500504 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.499069    5609 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 04 03:17:37.500504 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:17:37.499173    5609 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-128-50.us-west-2.compute.internal\" not found"
Nov 04 03:17:37.500504 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.499150    5609 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 03:11:26 +0000 UTC" deadline="2028-03-30 21:14:11.500005045 +0000 UTC"
Nov 04 03:17:37.500504 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.499626    5609 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12305h56m34.000384615s"
Nov 04 03:17:37.500504 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.499709    5609 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 04 03:17:37.500504 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.500353    5609 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 04 03:17:37.500504 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.500367    5609 factory.go:55] Registering systemd factory
Nov 04 03:17:37.500504 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.500377    5609 factory.go:223] Registration of the systemd container factory successfully
Nov 04 03:17:37.501113 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.500918    5609 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 04 03:17:37.501113 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.500951    5609 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 04 03:17:37.501330 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.501305    5609 factory.go:153] Registering CRI-O factory
Nov 04 03:17:37.501330 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.501327    5609 factory.go:223] Registration of the crio container factory successfully
Nov 04 03:17:37.501457 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.501346    5609 factory.go:103] Registering Raw factory
Nov 04 03:17:37.501457 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.501360    5609 manager.go:1199] Started watching for new ooms in manager
Nov 04 03:17:37.502175 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.501748    5609 manager.go:319] Starting recovery of all containers
Nov 04 03:17:37.504696 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.504633    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="21a42913-5c4e-4057-aae5-2804038f16e4" volumeName="kubernetes.io/configmap/21a42913-5c4e-4057-aae5-2804038f16e4-ovnkube-config" seLinuxMountContext=""
Nov 04 03:17:37.504765 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.504698    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6f627240-24ef-4aee-9810-51f422083ca3" volumeName="kubernetes.io/projected/6f627240-24ef-4aee-9810-51f422083ca3-kube-api-access-2kl5q" seLinuxMountContext=""
Nov 04 03:17:37.504765 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.504723    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aab0bd9b-cb90-4d9b-9f46-52c6c87a34cf" volumeName="kubernetes.io/projected/aab0bd9b-cb90-4d9b-9f46-52c6c87a34cf-kube-api-access-b7cm5" seLinuxMountContext=""
Nov 04 03:17:37.504765 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.504738    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fc3d3c0e-2877-4264-93a7-be3756d2dc5c" volumeName="kubernetes.io/projected/fc3d3c0e-2877-4264-93a7-be3756d2dc5c-kube-api-access-5krfn" seLinuxMountContext=""
Nov 04 03:17:37.504765 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.504752    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ff9f5afb-957c-4383-8fa9-698f7ada29b0" volumeName="kubernetes.io/configmap/ff9f5afb-957c-4383-8fa9-698f7ada29b0-whereabouts-flatfile-configmap" seLinuxMountContext=""
Nov 04 03:17:37.504906 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.504786    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="06698c23-887e-4ee5-9728-04098200c13d" volumeName="kubernetes.io/projected/06698c23-887e-4ee5-9728-04098200c13d-kube-api-access-v4sqp" seLinuxMountContext=""
Nov 04 03:17:37.504906 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.504807    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="536a0057-a438-4ad9-a4cd-b26cb7f3c645" volumeName="kubernetes.io/configmap/536a0057-a438-4ad9-a4cd-b26cb7f3c645-kube-rbac-proxy-cm" seLinuxMountContext=""
Nov 04 03:17:37.504906 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.504840    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6f627240-24ef-4aee-9810-51f422083ca3" volumeName="kubernetes.io/configmap/6f627240-24ef-4aee-9810-51f422083ca3-multus-daemon-config" seLinuxMountContext=""
Nov 04 03:17:37.504906 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.504869    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ff9f5afb-957c-4383-8fa9-698f7ada29b0" volumeName="kubernetes.io/projected/ff9f5afb-957c-4383-8fa9-698f7ada29b0-kube-api-access-x424d" seLinuxMountContext=""
Nov 04 03:17:37.504906 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.504894    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6dedccdc-aea2-4664-8480-ca3c9359437d" volumeName="kubernetes.io/configmap/6dedccdc-aea2-4664-8480-ca3c9359437d-config" seLinuxMountContext=""
Nov 04 03:17:37.504906 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.504905    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7713890e-763d-4a51-bffb-6cf8dd12fd18" volumeName="kubernetes.io/projected/7713890e-763d-4a51-bffb-6cf8dd12fd18-kube-api-access-hp4bb" seLinuxMountContext=""
Nov 04 03:17:37.505092 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.504929    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a683bd45-33c3-43fb-9530-292d7fea6970" volumeName="kubernetes.io/empty-dir/a683bd45-33c3-43fb-9530-292d7fea6970-etc-tuned" seLinuxMountContext=""
Nov 04 03:17:37.505092 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.504943    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="45621191-5080-4ee9-a08e-f7b2ee49fe41" volumeName="kubernetes.io/projected/45621191-5080-4ee9-a08e-f7b2ee49fe41-kube-api-access-6c8b9" seLinuxMountContext=""
Nov 04 03:17:37.505092 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.504956    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="029e7b7b-265e-459a-bc09-a90b5c179993" volumeName="kubernetes.io/secret/029e7b7b-265e-459a-bc09-a90b5c179993-agent-certs" seLinuxMountContext=""
Nov 04 03:17:37.505092 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.504981    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="068619f8-7fbb-4b22-ac7d-a1f144e084c0" volumeName="kubernetes.io/configmap/068619f8-7fbb-4b22-ac7d-a1f144e084c0-trusted-ca" seLinuxMountContext=""
Nov 04 03:17:37.505092 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.504994    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="068619f8-7fbb-4b22-ac7d-a1f144e084c0" volumeName="kubernetes.io/projected/068619f8-7fbb-4b22-ac7d-a1f144e084c0-kube-api-access-mcm4d" seLinuxMountContext=""
Nov 04 03:17:37.505092 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505005    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="21a42913-5c4e-4057-aae5-2804038f16e4" volumeName="kubernetes.io/configmap/21a42913-5c4e-4057-aae5-2804038f16e4-env-overrides" seLinuxMountContext=""
Nov 04 03:17:37.505092 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505031    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="068619f8-7fbb-4b22-ac7d-a1f144e084c0" volumeName="kubernetes.io/secret/068619f8-7fbb-4b22-ac7d-a1f144e084c0-serving-cert" seLinuxMountContext=""
Nov 04 03:17:37.505092 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505038    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0beb091e-5c2f-4136-93a2-771f9b861411" volumeName="kubernetes.io/configmap/0beb091e-5c2f-4136-93a2-771f9b861411-config" seLinuxMountContext=""
Nov 04 03:17:37.505092 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505045    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c1b21c41-4b51-4da8-8920-c38cf49554bf" volumeName="kubernetes.io/configmap/c1b21c41-4b51-4da8-8920-c38cf49554bf-service-ca-bundle" seLinuxMountContext=""
Nov 04 03:17:37.505092 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505075    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e87dded5-1e1b-45cb-ab2e-ae12550dfca7" volumeName="kubernetes.io/configmap/e87dded5-1e1b-45cb-ab2e-ae12550dfca7-telemetry-config" seLinuxMountContext=""
Nov 04 03:17:37.505092 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505086    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e87dded5-1e1b-45cb-ab2e-ae12550dfca7" volumeName="kubernetes.io/projected/e87dded5-1e1b-45cb-ab2e-ae12550dfca7-kube-api-access-sbrl5" seLinuxMountContext=""
Nov 04 03:17:37.505378 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505096    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f1f61b42-3b07-4892-90a7-2efa024e435b" volumeName="kubernetes.io/projected/f1f61b42-3b07-4892-90a7-2efa024e435b-registry-tls" seLinuxMountContext=""
Nov 04 03:17:37.505378 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505126    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fc612506-b699-44bc-a81b-06fee4e819fe" volumeName="kubernetes.io/secret/fc612506-b699-44bc-a81b-06fee4e819fe-image-registry-private-configuration" seLinuxMountContext=""
Nov 04 03:17:37.505378 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505141    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ff9f5afb-957c-4383-8fa9-698f7ada29b0" volumeName="kubernetes.io/configmap/ff9f5afb-957c-4383-8fa9-698f7ada29b0-cni-binary-copy" seLinuxMountContext=""
Nov 04 03:17:37.505378 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505163    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="04b955ef-528b-4573-b18f-0957658f79e4" volumeName="kubernetes.io/projected/04b955ef-528b-4573-b18f-0957658f79e4-kube-api-access-jltvv" seLinuxMountContext=""
Nov 04 03:17:37.505378 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505202    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0beb091e-5c2f-4136-93a2-771f9b861411" volumeName="kubernetes.io/projected/0beb091e-5c2f-4136-93a2-771f9b861411-kube-api-access-m4pj6" seLinuxMountContext=""
Nov 04 03:17:37.505378 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505211    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0beb091e-5c2f-4136-93a2-771f9b861411" volumeName="kubernetes.io/secret/0beb091e-5c2f-4136-93a2-771f9b861411-serving-cert" seLinuxMountContext=""
Nov 04 03:17:37.505378 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505261    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="029e7b7b-265e-459a-bc09-a90b5c179993" volumeName="kubernetes.io/configmap/029e7b7b-265e-459a-bc09-a90b5c179993-konnectivity-ca" seLinuxMountContext=""
Nov 04 03:17:37.505378 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505276    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="068619f8-7fbb-4b22-ac7d-a1f144e084c0" volumeName="kubernetes.io/configmap/068619f8-7fbb-4b22-ac7d-a1f144e084c0-config" seLinuxMountContext=""
Nov 04 03:17:37.505378 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505283    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3b178ce9-e140-4420-81f1-627de00b90c6" volumeName="kubernetes.io/projected/3b178ce9-e140-4420-81f1-627de00b90c6-kube-api-access-ghplp" seLinuxMountContext=""
Nov 04 03:17:37.505378 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505290    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="55c4af4c-188b-495b-9052-5700a1f60602" volumeName="kubernetes.io/secret/55c4af4c-188b-495b-9052-5700a1f60602-cert" seLinuxMountContext=""
Nov 04 03:17:37.505378 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505332    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6dedccdc-aea2-4664-8480-ca3c9359437d" volumeName="kubernetes.io/projected/6dedccdc-aea2-4664-8480-ca3c9359437d-kube-api-access-5mbsz" seLinuxMountContext=""
Nov 04 03:17:37.505378 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505370    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e487dae4-8ac9-4acd-a8ad-a0d149909475" volumeName="kubernetes.io/secret/e487dae4-8ac9-4acd-a8ad-a0d149909475-samples-operator-tls" seLinuxMountContext=""
Nov 04 03:17:37.505378 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505378    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f1f61b42-3b07-4892-90a7-2efa024e435b" volumeName="kubernetes.io/empty-dir/f1f61b42-3b07-4892-90a7-2efa024e435b-ca-trust-extracted" seLinuxMountContext=""
Nov 04 03:17:37.505768 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505411    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f1f61b42-3b07-4892-90a7-2efa024e435b" volumeName="kubernetes.io/projected/f1f61b42-3b07-4892-90a7-2efa024e435b-bound-sa-token" seLinuxMountContext=""
Nov 04 03:17:37.505768 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505422    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3b178ce9-e140-4420-81f1-627de00b90c6" volumeName="kubernetes.io/empty-dir/3b178ce9-e140-4420-81f1-627de00b90c6-snapshots" seLinuxMountContext=""
Nov 04 03:17:37.505768 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505429    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f1f61b42-3b07-4892-90a7-2efa024e435b" volumeName="kubernetes.io/projected/f1f61b42-3b07-4892-90a7-2efa024e435b-kube-api-access-kjtnj" seLinuxMountContext=""
Nov 04 03:17:37.505768 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505450    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f1f61b42-3b07-4892-90a7-2efa024e435b" volumeName="kubernetes.io/secret/f1f61b42-3b07-4892-90a7-2efa024e435b-image-registry-private-configuration" seLinuxMountContext=""
Nov 04 03:17:37.505768 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505458    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fc612506-b699-44bc-a81b-06fee4e819fe" volumeName="kubernetes.io/configmap/fc612506-b699-44bc-a81b-06fee4e819fe-trusted-ca" seLinuxMountContext=""
Nov 04 03:17:37.505768 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505464    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fc612506-b699-44bc-a81b-06fee4e819fe" volumeName="kubernetes.io/projected/fc612506-b699-44bc-a81b-06fee4e819fe-registry-tls" seLinuxMountContext=""
Nov 04 03:17:37.505768 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505492    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="06698c23-887e-4ee5-9728-04098200c13d" volumeName="kubernetes.io/empty-dir/06698c23-887e-4ee5-9728-04098200c13d-tmp-dir" seLinuxMountContext=""
Nov 04 03:17:37.505768 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505527    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="21a42913-5c4e-4057-aae5-2804038f16e4" volumeName="kubernetes.io/projected/21a42913-5c4e-4057-aae5-2804038f16e4-kube-api-access-9trx7" seLinuxMountContext=""
Nov 04 03:17:37.505768 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505537    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3b178ce9-e140-4420-81f1-627de00b90c6" volumeName="kubernetes.io/configmap/3b178ce9-e140-4420-81f1-627de00b90c6-service-ca-bundle" seLinuxMountContext=""
Nov 04 03:17:37.505768 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505560    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="45621191-5080-4ee9-a08e-f7b2ee49fe41" volumeName="kubernetes.io/configmap/45621191-5080-4ee9-a08e-f7b2ee49fe41-iptables-alerter-script" seLinuxMountContext=""
Nov 04 03:17:37.505768 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505569    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7713890e-763d-4a51-bffb-6cf8dd12fd18" volumeName="kubernetes.io/configmap/7713890e-763d-4a51-bffb-6cf8dd12fd18-serviceca" seLinuxMountContext=""
Nov 04 03:17:37.505768 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505575    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aab0bd9b-cb90-4d9b-9f46-52c6c87a34cf" volumeName="kubernetes.io/empty-dir/aab0bd9b-cb90-4d9b-9f46-52c6c87a34cf-tmp-dir" seLinuxMountContext=""
Nov 04 03:17:37.505768 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505590    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e87dded5-1e1b-45cb-ab2e-ae12550dfca7" volumeName="kubernetes.io/secret/e87dded5-1e1b-45cb-ab2e-ae12550dfca7-cluster-monitoring-operator-tls" seLinuxMountContext=""
Nov 04 03:17:37.505768 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505597    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fc612506-b699-44bc-a81b-06fee4e819fe" volumeName="kubernetes.io/empty-dir/fc612506-b699-44bc-a81b-06fee4e819fe-ca-trust-extracted" seLinuxMountContext=""
Nov 04 03:17:37.505768 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505613    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c1b21c41-4b51-4da8-8920-c38cf49554bf" volumeName="kubernetes.io/secret/c1b21c41-4b51-4da8-8920-c38cf49554bf-metrics-certs" seLinuxMountContext=""
Nov 04 03:17:37.505768 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505628    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fc612506-b699-44bc-a81b-06fee4e819fe" volumeName="kubernetes.io/secret/fc612506-b699-44bc-a81b-06fee4e819fe-installation-pull-secrets" seLinuxMountContext=""
Nov 04 03:17:37.505768 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505636    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3b178ce9-e140-4420-81f1-627de00b90c6" volumeName="kubernetes.io/empty-dir/3b178ce9-e140-4420-81f1-627de00b90c6-tmp" seLinuxMountContext=""
Nov 04 03:17:37.505768 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505645    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f1f61b42-3b07-4892-90a7-2efa024e435b" volumeName="kubernetes.io/secret/f1f61b42-3b07-4892-90a7-2efa024e435b-installation-pull-secrets" seLinuxMountContext=""
Nov 04 03:17:37.505768 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505661    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6f627240-24ef-4aee-9810-51f422083ca3" volumeName="kubernetes.io/configmap/6f627240-24ef-4aee-9810-51f422083ca3-cni-binary-copy" seLinuxMountContext=""
Nov 04 03:17:37.505768 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505671    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a683bd45-33c3-43fb-9530-292d7fea6970" volumeName="kubernetes.io/empty-dir/a683bd45-33c3-43fb-9530-292d7fea6970-tmp" seLinuxMountContext=""
Nov 04 03:17:37.505768 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505680    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c1b21c41-4b51-4da8-8920-c38cf49554bf" volumeName="kubernetes.io/secret/c1b21c41-4b51-4da8-8920-c38cf49554bf-stats-auth" seLinuxMountContext=""
Nov 04 03:17:37.505768 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505737    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cc469246-2293-4301-9e46-bf9d609be964" volumeName="kubernetes.io/projected/cc469246-2293-4301-9e46-bf9d609be964-kube-api-access-cnrbg" seLinuxMountContext=""
Nov 04 03:17:37.505768 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505760    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="06698c23-887e-4ee5-9728-04098200c13d" volumeName="kubernetes.io/configmap/06698c23-887e-4ee5-9728-04098200c13d-config-volume" seLinuxMountContext=""
Nov 04 03:17:37.505768 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505776    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="06698c23-887e-4ee5-9728-04098200c13d" volumeName="kubernetes.io/secret/06698c23-887e-4ee5-9728-04098200c13d-metrics-tls" seLinuxMountContext=""
Nov 04 03:17:37.506449 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505797    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="536a0057-a438-4ad9-a4cd-b26cb7f3c645" volumeName="kubernetes.io/projected/536a0057-a438-4ad9-a4cd-b26cb7f3c645-kube-api-access-7fhxt" seLinuxMountContext=""
Nov 04 03:17:37.506449 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505807    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e0c72601-2624-4632-9f15-c19fcc489aae" volumeName="kubernetes.io/projected/e0c72601-2624-4632-9f15-c19fcc489aae-kube-api-access-66hms" seLinuxMountContext=""
Nov 04 03:17:37.506449 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505815    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="10cd06d4-ef1a-4ecf-a72c-e6d0084f3032" volumeName="kubernetes.io/projected/10cd06d4-ef1a-4ecf-a72c-e6d0084f3032-kube-api-access-597ml" seLinuxMountContext=""
Nov 04 03:17:37.506449 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505836    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6dedccdc-aea2-4664-8480-ca3c9359437d" volumeName="kubernetes.io/secret/6dedccdc-aea2-4664-8480-ca3c9359437d-serving-cert" seLinuxMountContext=""
Nov 04 03:17:37.506449 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505851    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a683bd45-33c3-43fb-9530-292d7fea6970" volumeName="kubernetes.io/projected/a683bd45-33c3-43fb-9530-292d7fea6970-kube-api-access-qm4f8" seLinuxMountContext=""
Nov 04 03:17:37.506449 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505877    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c1b21c41-4b51-4da8-8920-c38cf49554bf" volumeName="kubernetes.io/secret/c1b21c41-4b51-4da8-8920-c38cf49554bf-default-certificate" seLinuxMountContext=""
Nov 04 03:17:37.506449 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505923    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d10e9268-df07-428b-ae25-528d9fe075d2" volumeName="kubernetes.io/secret/d10e9268-df07-428b-ae25-528d9fe075d2-original-pull-secret" seLinuxMountContext=""
Nov 04 03:17:37.506449 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505964    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d731646d-23ab-42d7-ae48-974929b3d2dc" volumeName="kubernetes.io/configmap/d731646d-23ab-42d7-ae48-974929b3d2dc-nginx-conf" seLinuxMountContext=""
Nov 04 03:17:37.506449 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.505977    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f1f61b42-3b07-4892-90a7-2efa024e435b" volumeName="kubernetes.io/configmap/f1f61b42-3b07-4892-90a7-2efa024e435b-registry-certificates" seLinuxMountContext=""
Nov 04 03:17:37.506449 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.506004    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fc612506-b699-44bc-a81b-06fee4e819fe" volumeName="kubernetes.io/configmap/fc612506-b699-44bc-a81b-06fee4e819fe-registry-certificates" seLinuxMountContext=""
Nov 04 03:17:37.506449 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.506016    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3b178ce9-e140-4420-81f1-627de00b90c6" volumeName="kubernetes.io/configmap/3b178ce9-e140-4420-81f1-627de00b90c6-trusted-ca-bundle" seLinuxMountContext=""
Nov 04 03:17:37.506449 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.506026    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3b178ce9-e140-4420-81f1-627de00b90c6" volumeName="kubernetes.io/secret/3b178ce9-e140-4420-81f1-627de00b90c6-serving-cert" seLinuxMountContext=""
Nov 04 03:17:37.506449 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.506066    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="55c4af4c-188b-495b-9052-5700a1f60602" volumeName="kubernetes.io/projected/55c4af4c-188b-495b-9052-5700a1f60602-kube-api-access-5h469" seLinuxMountContext=""
Nov 04 03:17:37.506449 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.506077    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c1b21c41-4b51-4da8-8920-c38cf49554bf" volumeName="kubernetes.io/projected/c1b21c41-4b51-4da8-8920-c38cf49554bf-kube-api-access-fsjms" seLinuxMountContext=""
Nov 04 03:17:37.506449 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.506114    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d731646d-23ab-42d7-ae48-974929b3d2dc" volumeName="kubernetes.io/secret/d731646d-23ab-42d7-ae48-974929b3d2dc-networking-console-plugin-cert" seLinuxMountContext=""
Nov 04 03:17:37.506449 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.506164    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e487dae4-8ac9-4acd-a8ad-a0d149909475" volumeName="kubernetes.io/configmap/e487dae4-8ac9-4acd-a8ad-a0d149909475-samples-operator-config" seLinuxMountContext=""
Nov 04 03:17:37.506449 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.506187    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e487dae4-8ac9-4acd-a8ad-a0d149909475" volumeName="kubernetes.io/projected/e487dae4-8ac9-4acd-a8ad-a0d149909475-kube-api-access-vlxxz" seLinuxMountContext=""
Nov 04 03:17:37.506449 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.506228    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f1f61b42-3b07-4892-90a7-2efa024e435b" volumeName="kubernetes.io/configmap/f1f61b42-3b07-4892-90a7-2efa024e435b-trusted-ca" seLinuxMountContext=""
Nov 04 03:17:37.506449 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.506260    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fc3d3c0e-2877-4264-93a7-be3756d2dc5c" volumeName="kubernetes.io/secret/fc3d3c0e-2877-4264-93a7-be3756d2dc5c-metrics-certs" seLinuxMountContext=""
Nov 04 03:17:37.506449 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.506279    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fc612506-b699-44bc-a81b-06fee4e819fe" volumeName="kubernetes.io/projected/fc612506-b699-44bc-a81b-06fee4e819fe-bound-sa-token" seLinuxMountContext=""
Nov 04 03:17:37.506449 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.506295    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fc612506-b699-44bc-a81b-06fee4e819fe" volumeName="kubernetes.io/projected/fc612506-b699-44bc-a81b-06fee4e819fe-kube-api-access-bfc98" seLinuxMountContext=""
Nov 04 03:17:37.506449 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.506332    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="21a42913-5c4e-4057-aae5-2804038f16e4" volumeName="kubernetes.io/secret/21a42913-5c4e-4057-aae5-2804038f16e4-ovn-node-metrics-cert" seLinuxMountContext=""
Nov 04 03:17:37.506449 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.506366    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d604b334-07f4-40f2-bfa9-4b26c301c84c" volumeName="kubernetes.io/projected/d604b334-07f4-40f2-bfa9-4b26c301c84c-kube-api-access-n5cdl" seLinuxMountContext=""
Nov 04 03:17:37.506449 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.506385    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e451b329-bd05-44f5-9544-8283f76df9d6" volumeName="kubernetes.io/projected/e451b329-bd05-44f5-9544-8283f76df9d6-kube-api-access-8g8qf" seLinuxMountContext=""
Nov 04 03:17:37.506449 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.506453    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ff9f5afb-957c-4383-8fa9-698f7ada29b0" volumeName="kubernetes.io/configmap/ff9f5afb-957c-4383-8fa9-698f7ada29b0-cni-sysctl-allowlist" seLinuxMountContext=""
Nov 04 03:17:37.507126 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.507035    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="21a42913-5c4e-4057-aae5-2804038f16e4" volumeName="kubernetes.io/configmap/21a42913-5c4e-4057-aae5-2804038f16e4-ovnkube-script-lib" seLinuxMountContext=""
Nov 04 03:17:37.507126 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.507051    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="536a0057-a438-4ad9-a4cd-b26cb7f3c645" volumeName="kubernetes.io/empty-dir/536a0057-a438-4ad9-a4cd-b26cb7f3c645-data-volume" seLinuxMountContext=""
Nov 04 03:17:37.507126 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.507074    5609 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a51f26a7-0ec8-4c56-881a-98c6d625478b" volumeName="kubernetes.io/projected/a51f26a7-0ec8-4c56-881a-98c6d625478b-kube-api-access-2ldp7" seLinuxMountContext=""
Nov 04 03:17:37.507126 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.507088    5609 reconstruct.go:97] "Volume reconstruction finished"
Nov 04 03:17:37.507126 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.507096    5609 reconciler.go:29] "Reconciler: start to sync state"
Nov 04 03:17:37.517977 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.517951    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-p9989_068619f8-7fbb-4b22-ac7d-a1f144e084c0/console-operator/2.log"
Nov 04 03:17:37.518221 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.518206    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-p9989_068619f8-7fbb-4b22-ac7d-a1f144e084c0/console-operator/2.log"
Nov 04 03:17:37.524849 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.524811    5609 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 04 03:17:37.526946 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.526901    5609 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 04 03:17:37.526946 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.526928    5609 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 04 03:17:37.526946 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.526947    5609 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 04 03:17:37.527260 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:17:37.526985    5609 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Nov 04 03:17:37.528222 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.528197    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vmw5z_21a42913-5c4e-4057-aae5-2804038f16e4/ovn-acl-logging/0.log"
Nov 04 03:17:37.528334 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.528316    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vmw5z_21a42913-5c4e-4057-aae5-2804038f16e4/ovn-acl-logging/0.log"
Nov 04 03:17:37.543933 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.543894    5609 generic.go:356] "Generic (PLEG): container finished" podID="ff9f5afb-957c-4383-8fa9-698f7ada29b0" containerID="945d7e1f5ca47bb2972f8b9e6ea4ad590723f0f5ebd6d6e7999fffe8d5a9d192" exitCode=0
Nov 04 03:17:37.543933 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.543933    5609 generic.go:356] "Generic (PLEG): container finished" podID="ff9f5afb-957c-4383-8fa9-698f7ada29b0" containerID="ea1737077340b39314b28c31e58d589013d2fe974ca30aa9904407df7b66a060" exitCode=0
Nov 04 03:17:37.544163 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.543945    5609 generic.go:356] "Generic (PLEG): container finished" podID="ff9f5afb-957c-4383-8fa9-698f7ada29b0" containerID="5e920a96493e2c1f6282e9575be01da804c9971f9cf7815cd96b92e93427e4fd" exitCode=0
Nov 04 03:17:37.544163 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.543957    5609 generic.go:356] "Generic (PLEG): container finished" podID="ff9f5afb-957c-4383-8fa9-698f7ada29b0" containerID="2d4e3a80926cb5afa449bd6841b511ab58c9bb3dfa1a52e63ea8ed68fbad7e12" exitCode=0
Nov 04 03:17:37.544163 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.543966    5609 generic.go:356] "Generic (PLEG): container finished" podID="ff9f5afb-957c-4383-8fa9-698f7ada29b0" containerID="709c6f80267e0ae83963c9133ee84eb003d734bf58d764285174ddf89e8378de" exitCode=0
Nov 04 03:17:37.544163 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.543977    5609 generic.go:356] "Generic (PLEG): container finished" podID="ff9f5afb-957c-4383-8fa9-698f7ada29b0" containerID="e6471b52d954d3714e88a74c51f99cdde0ce2d311927d8583d0f2d12a37f3e9d" exitCode=0
Nov 04 03:17:37.546060 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.546032    5609 generic.go:356] "Generic (PLEG): container finished" podID="2f2b595dbb5c94185c06c3dd3c0c8dce" containerID="24bc55a0ebc85a2d139d06103d000aa6c926738a73bf7f80c7cd444f5253a450" exitCode=0
Nov 04 03:17:37.559773 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.559746    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vmw5z_21a42913-5c4e-4057-aae5-2804038f16e4/ovn-acl-logging/0.log"
Nov 04 03:17:37.560221 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.560185    5609 generic.go:356] "Generic (PLEG): container finished" podID="21a42913-5c4e-4057-aae5-2804038f16e4" containerID="2f44e914fdba9a07bd980e85975eb79d8885d9caba161a8263e0dce9125701b1" exitCode=1
Nov 04 03:17:37.578632 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.578606    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-p9989_068619f8-7fbb-4b22-ac7d-a1f144e084c0/console-operator/2.log"
Nov 04 03:17:37.578632 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.578635    5609 generic.go:356] "Generic (PLEG): container finished" podID="068619f8-7fbb-4b22-ac7d-a1f144e084c0" containerID="92d2557ce0ed61ad88da7d5c6d76d645bf407cd2ce7733bd5f621f087bce4578" exitCode=255
Nov 04 03:17:37.599253 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:17:37.599228    5609 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-128-50.us-west-2.compute.internal\" not found"
Nov 04 03:17:37.610641 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.610616    5609 manager.go:324] Recovery completed
Nov 04 03:17:37.627548 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:17:37.627457    5609 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet"
Nov 04 03:17:37.630077 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.630057    5609 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 03:17:37.632029 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.632013    5609 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-50.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 03:17:37.632123 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.632047    5609 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-50.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 03:17:37.632123 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.632062    5609 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-50.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 03:17:37.634948 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.634927    5609 cpu_manager.go:226] "Starting" policy="none"
Nov 04 03:17:37.634948 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.634941    5609 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 04 03:17:37.635083 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.634952    5609 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 03:17:37.635131 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.634959    5609 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 04 03:17:37.635249 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.635232    5609 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Nov 04 03:17:37.635294 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.635248    5609 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Nov 04 03:17:37.635294 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.635279    5609 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Nov 04 03:17:37.635294 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.635285    5609 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Nov 04 03:17:37.635294 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.635291    5609 policy_none.go:50] "Start"
Nov 04 03:17:37.635419 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.635300    5609 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 04 03:17:37.635419 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.635310    5609 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 04 03:17:37.636137 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.636118    5609 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-50.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 03:17:37.636180 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.636155    5609 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-50.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 03:17:37.636180 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.636168    5609 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-50.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 03:17:37.637020 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.636998    5609 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Nov 04 03:17:37.637060 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.637022    5609 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Nov 04 03:17:37.637060 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.637029    5609 policy_none.go:44] "Start"
Nov 04 03:17:37.643122 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.643102    5609 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 03:17:37.644131 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.644106    5609 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 04 03:17:37.646884 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.646871    5609 manager.go:341] "Starting Device Plugin manager"
Nov 04 03:17:37.646974 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:17:37.646897    5609 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 04 03:17:37.646974 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.646906    5609 server.go:87] "Starting device plugin registration server"
Nov 04 03:17:37.647247 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.647233    5609 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 04 03:17:37.647292 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.647249    5609 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 04 03:17:37.647389 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.647370    5609 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 04 03:17:37.647534 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.647495    5609 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 04 03:17:37.647639 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.647539    5609 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 04 03:17:37.647639 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.647545    5609 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 04 03:17:37.648181 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.648094    5609 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-04T03:17:37.647527435Z","UUID":"fb330492-defb-420e-b704-bb76bdb798d4","Handler":null,"Name":"","Endpoint":""}
Nov 04 03:17:37.649340 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:17:37.649323    5609 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 04 03:17:37.649514 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:17:37.649358    5609 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-50.us-west-2.compute.internal\" not found"
Nov 04 03:17:37.650438 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.650420    5609 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 04 03:17:37.650531 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.650446    5609 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 04 03:17:37.690241 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.690208    5609 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 03:17:37.690241 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.690238    5609 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Nov 04 03:17:37.737708 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.737671    5609 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 03:17:37.747964 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.747938    5609 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 03:17:37.749072 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.749051    5609 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-50.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 03:17:37.749160 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.749087    5609 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-50.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 03:17:37.749160 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.749098    5609 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-50.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 03:17:37.749160 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.749143    5609 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-128-50.us-west-2.compute.internal"
Nov 04 03:17:37.783693 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.783648    5609 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 03:17:37.828488 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.828390    5609 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.us-west-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-50.us-west-2.compute.internal"]
Nov 04 03:17:37.829108 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.829077    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.us-west-2.compute.internal" event={"ID":"2f2b595dbb5c94185c06c3dd3c0c8dce","Type":"ContainerStarted","Data":"cb37a668c8ff0efe8e16b948c8e5073c8e537c8b73c1fdccdbaa3570ac9260fb"}
Nov 04 03:17:37.829147 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.829120    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.us-west-2.compute.internal" event={"ID":"2f2b595dbb5c94185c06c3dd3c0c8dce","Type":"ContainerDied","Data":"24bc55a0ebc85a2d139d06103d000aa6c926738a73bf7f80c7cd444f5253a450"}
Nov 04 03:17:37.829147 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.829140    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.us-west-2.compute.internal" event={"ID":"2f2b595dbb5c94185c06c3dd3c0c8dce","Type":"ContainerStarted","Data":"18e250ae36ea4dcd84c44211ccde315ef072c36d75a4208979550f27a5d728f2"}
Nov 04 03:17:37.829241 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.829230    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-50.us-west-2.compute.internal" event={"ID":"aeb0f7512125881a2a5e1813eb42a8f7","Type":"ContainerStarted","Data":"9e920e155b78915b58a88d5cd95c09eb1a4ac7057fb6e0b810e2ac157e5defee"}
Nov 04 03:17:37.829275 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.829248    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-50.us-west-2.compute.internal" event={"ID":"aeb0f7512125881a2a5e1813eb42a8f7","Type":"ContainerStarted","Data":"5e0193cee68746668d04cd45c7dbfb28ff8d5a48f555b3213f1549e560278900"}
Nov 04 03:17:37.829317 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.829303    5609 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-50.us-west-2.compute.internal"
Nov 04 03:17:37.829348 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.829314    5609 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.us-west-2.compute.internal"
Nov 04 03:17:37.852649 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.852621    5609 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-128-50.us-west-2.compute.internal"
Nov 04 03:17:37.852725 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.852699    5609 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-128-50.us-west-2.compute.internal"
Nov 04 03:17:37.887643 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.887573    5609 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 03:17:37.887643 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:17:37.887629    5609 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-128-50.us-west-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-50.us-west-2.compute.internal"
Nov 04 03:17:37.887813 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.887795    5609 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 03:17:37.887853 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:17:37.887820    5609 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-128-50.us-west-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.us-west-2.compute.internal"
Nov 04 03:17:37.991134 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.991095    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2f2b595dbb5c94185c06c3dd3c0c8dce-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-50.us-west-2.compute.internal\" (UID: \"2f2b595dbb5c94185c06c3dd3c0c8dce\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.us-west-2.compute.internal"
Nov 04 03:17:37.991134 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.991140    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2f2b595dbb5c94185c06c3dd3c0c8dce-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-50.us-west-2.compute.internal\" (UID: \"2f2b595dbb5c94185c06c3dd3c0c8dce\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.us-west-2.compute.internal"
Nov 04 03:17:37.991322 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:37.991161    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/aeb0f7512125881a2a5e1813eb42a8f7-config\") pod \"kube-apiserver-proxy-ip-10-0-128-50.us-west-2.compute.internal\" (UID: \"aeb0f7512125881a2a5e1813eb42a8f7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-50.us-west-2.compute.internal"
Nov 04 03:17:38.091414 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.091357    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2f2b595dbb5c94185c06c3dd3c0c8dce-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-50.us-west-2.compute.internal\" (UID: \"2f2b595dbb5c94185c06c3dd3c0c8dce\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.us-west-2.compute.internal"
Nov 04 03:17:38.091414 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.091378    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2f2b595dbb5c94185c06c3dd3c0c8dce-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-50.us-west-2.compute.internal\" (UID: \"2f2b595dbb5c94185c06c3dd3c0c8dce\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.us-west-2.compute.internal"
Nov 04 03:17:38.091608 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.091447    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2f2b595dbb5c94185c06c3dd3c0c8dce-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-50.us-west-2.compute.internal\" (UID: \"2f2b595dbb5c94185c06c3dd3c0c8dce\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.us-west-2.compute.internal"
Nov 04 03:17:38.091608 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.091469    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/aeb0f7512125881a2a5e1813eb42a8f7-config\") pod \"kube-apiserver-proxy-ip-10-0-128-50.us-west-2.compute.internal\" (UID: \"aeb0f7512125881a2a5e1813eb42a8f7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-50.us-west-2.compute.internal"
Nov 04 03:17:38.091608 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.091502    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/aeb0f7512125881a2a5e1813eb42a8f7-config\") pod \"kube-apiserver-proxy-ip-10-0-128-50.us-west-2.compute.internal\" (UID: \"aeb0f7512125881a2a5e1813eb42a8f7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-50.us-west-2.compute.internal"
Nov 04 03:17:38.091608 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.091570    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2f2b595dbb5c94185c06c3dd3c0c8dce-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-50.us-west-2.compute.internal\" (UID: \"2f2b595dbb5c94185c06c3dd3c0c8dce\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.us-west-2.compute.internal"
Nov 04 03:17:38.485456 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.485389    5609 apiserver.go:52] "Watching apiserver"
Nov 04 03:17:38.586028 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.584386    5609 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 04 03:17:38.586604 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.586572    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-67bjk","openshift-multus/network-metrics-daemon-lngrr","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.us-west-2.compute.internal","openshift-service-ca-operator/service-ca-operator-544d749c8c-js5xd","openshift-monitoring/cluster-monitoring-operator-884576c57-gmt9g","openshift-ovn-kubernetes/ovnkube-node-vmw5z","openshift-dns/node-resolver-6pwhj","openshift-network-diagnostics/network-check-target-hjssc","openshift-network-console/networking-console-plugin-86c478bb88-tlhng","openshift-dns/dns-default-s4852","kube-system/kas-connection-checker-685895c7c9-n6r7s","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8","kube-system/konnectivity-agent-m2vkr","openshift-image-registry/node-ca-n4b5s","openshift-insights/insights-operator-7c4986d449-hsb49","openshift-network-operator/iptables-alerter-sdhv4","openshift-ingress/router-default-7cfb798487-kbfzk","openshift-image-registry/image-registry-79d6dd87ff-n2jnc","openshift-cluster-node-tuning-operator/tuned-qtt77","openshift-ingress-canary/ingress-canary-qbskg","openshift-console-operator/console-operator-7cd56b8dbc-p9989","kube-system/kube-apiserver-proxy-ip-10-0-128-50.us-west-2.compute.internal","kube-system/kas-connection-checker-685895c7c9-xpmwt","kube-system/kas-connection-checker-685895c7c9-mdchx","kube-system/global-pull-secret-syncer-tkkhk","openshift-insights/insights-runtime-extractor-46btb","openshift-multus/multus-9xgrv","openshift-network-diagnostics/network-check-source-6f78d59f99-7lkrx","openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dln2k","openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-6c2jp","openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-4z8fp","openshift-image-registry/image-registry-7cb9f7c89d-wnpll"]
Nov 04 03:17:38.599456 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.599426    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-46btb"
Nov 04 03:17:38.599714 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.599702    5609 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-50.us-west-2.compute.internal"
Nov 04 03:17:38.599795 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.599779    5609 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.us-west-2.compute.internal"
Nov 04 03:17:38.601449 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.601428    5609 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 04 03:17:38.638011 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.637473    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Nov 04 03:17:38.638011 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.637564    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Nov 04 03:17:38.638011 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.637469    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Nov 04 03:17:38.638011 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.637635    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Nov 04 03:17:38.638366 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.638342    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Nov 04 03:17:38.638968 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.638947    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Nov 04 03:17:38.639058 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.639038    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Nov 04 03:17:38.639224 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.639207    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Nov 04 03:17:38.639929 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.639910    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 04 03:17:38.640022 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.640012    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 04 03:17:38.640194 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.640176    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Nov 04 03:17:38.640317 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.640303    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Nov 04 03:17:38.640876 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.640852    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Nov 04 03:17:38.641159 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.641139    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Nov 04 03:17:38.641250 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.641229    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-f4sxh\""
Nov 04 03:17:38.641250 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.641236    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 04 03:17:38.641424 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.641388    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Nov 04 03:17:38.641424 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.641421    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-2nc4r\""
Nov 04 03:17:38.641531 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.641431    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Nov 04 03:17:38.641531 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.641465    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Nov 04 03:17:38.641531 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.641499    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Nov 04 03:17:38.641531 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.641526    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Nov 04 03:17:38.641708 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.641585    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 04 03:17:38.641708 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.641595    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Nov 04 03:17:38.641708 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.641605    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 04 03:17:38.641708 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.641627    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Nov 04 03:17:38.641708 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.641651    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Nov 04 03:17:38.641708 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.641656    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 04 03:17:38.641708 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.641688    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Nov 04 03:17:38.642003 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.641716    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Nov 04 03:17:38.642003 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.641737    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Nov 04 03:17:38.642199 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.642172    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Nov 04 03:17:38.642199 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.642179    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-2z9f6\""
Nov 04 03:17:38.642351 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.642251    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Nov 04 03:17:38.642351 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.642316    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Nov 04 03:17:38.642351 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.642342    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Nov 04 03:17:38.642467 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.642364    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Nov 04 03:17:38.642498 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.642481    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Nov 04 03:17:38.642539 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.642523    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Nov 04 03:17:38.642579 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.642563    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Nov 04 03:17:38.642920 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.642634    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Nov 04 03:17:38.685974 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.685938    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Nov 04 03:17:38.685974 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.685960    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Nov 04 03:17:38.688975 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.688952    5609 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 03:17:38.689056 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:17:38.689005    5609 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-128-50.us-west-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-50.us-west-2.compute.internal"
Nov 04 03:17:38.689323 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.689308    5609 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 03:17:38.689389 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:17:38.689358    5609 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-128-50.us-west-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.us-west-2.compute.internal"
Nov 04 03:17:38.695642 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.695618    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1b21c41-4b51-4da8-8920-c38cf49554bf-metrics-certs\") pod \"router-default-7cfb798487-kbfzk\" (UID: \"c1b21c41-4b51-4da8-8920-c38cf49554bf\") " pod="openshift-ingress/router-default-7cfb798487-kbfzk"
Nov 04 03:17:38.695746 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.695655    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-host-slash\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.695746 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.695681    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-run-openvswitch\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.695838 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.695778    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/029e7b7b-265e-459a-bc09-a90b5c179993-agent-certs\") pod \"konnectivity-agent-m2vkr\" (UID: \"029e7b7b-265e-459a-bc09-a90b5c179993\") " pod="kube-system/konnectivity-agent-m2vkr"
Nov 04 03:17:38.695838 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.695812    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnrbg\" (UniqueName: \"kubernetes.io/projected/cc469246-2293-4301-9e46-bf9d609be964-kube-api-access-cnrbg\") pod \"network-check-source-6f78d59f99-7lkrx\" (UID: \"cc469246-2293-4301-9e46-bf9d609be964\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-7lkrx"
Nov 04 03:17:38.695838 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.695831    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-node-log\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.695957 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.695855    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-host-cni-bin\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.695957 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.695933    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1b21c41-4b51-4da8-8920-c38cf49554bf-metrics-certs\") pod \"router-default-7cfb798487-kbfzk\" (UID: \"c1b21c41-4b51-4da8-8920-c38cf49554bf\") " pod="openshift-ingress/router-default-7cfb798487-kbfzk"
Nov 04 03:17:38.696048 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.696011    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.696103 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.696047    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8g8qf\" (UniqueName: \"kubernetes.io/projected/e451b329-bd05-44f5-9544-8283f76df9d6-kube-api-access-8g8qf\") pod \"volume-data-source-validator-58c9f5f46d-6c2jp\" (UID: \"e451b329-bd05-44f5-9544-8283f76df9d6\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-6c2jp"
Nov 04 03:17:38.696103 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.696078    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aab0bd9b-cb90-4d9b-9f46-52c6c87a34cf-tmp-dir\") pod \"node-resolver-6pwhj\" (UID: \"aab0bd9b-cb90-4d9b-9f46-52c6c87a34cf\") " pod="openshift-dns/node-resolver-6pwhj"
Nov 04 03:17:38.696198 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.696120    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlxxz\" (UniqueName: \"kubernetes.io/projected/e487dae4-8ac9-4acd-a8ad-a0d149909475-kube-api-access-vlxxz\") pod \"cluster-samples-operator-6ddb6c8b94-4z8fp\" (UID: \"e487dae4-8ac9-4acd-a8ad-a0d149909475\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-4z8fp"
Nov 04 03:17:38.696198 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.696159    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-log-socket\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.696198 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.696161    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aab0bd9b-cb90-4d9b-9f46-52c6c87a34cf-tmp-dir\") pod \"node-resolver-6pwhj\" (UID: \"aab0bd9b-cb90-4d9b-9f46-52c6c87a34cf\") " pod="openshift-dns/node-resolver-6pwhj"
Nov 04 03:17:38.696304 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.696219    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/536a0057-a438-4ad9-a4cd-b26cb7f3c645-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-46btb\" (UID: \"536a0057-a438-4ad9-a4cd-b26cb7f3c645\") " pod="openshift-insights/insights-runtime-extractor-46btb"
Nov 04 03:17:38.696304 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.696248    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6f627240-24ef-4aee-9810-51f422083ca3-multus-daemon-config\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:17:38.696304 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.696277    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-etc-openvswitch\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.696424 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.696333    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f1f61b42-3b07-4892-90a7-2efa024e435b-registry-tls\") pod \"image-registry-79d6dd87ff-n2jnc\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") " pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:17:38.696424 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.696357    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-var-lib-kubelet\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:17:38.696424 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.696374    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc3d3c0e-2877-4264-93a7-be3756d2dc5c-metrics-certs\") pod \"network-metrics-daemon-lngrr\" (UID: \"fc3d3c0e-2877-4264-93a7-be3756d2dc5c\") " pod="openshift-multus/network-metrics-daemon-lngrr"
Nov 04 03:17:38.696424 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.696409    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-multus-socket-dir-parent\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:17:38.696586 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.696427    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3b178ce9-e140-4420-81f1-627de00b90c6-serving-cert\") pod \"insights-operator-7c4986d449-hsb49\" (UID: \"3b178ce9-e140-4420-81f1-627de00b90c6\") " pod="openshift-insights/insights-operator-7c4986d449-hsb49"
Nov 04 03:17:38.696586 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.696442    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/45621191-5080-4ee9-a08e-f7b2ee49fe41-host-slash\") pod \"iptables-alerter-sdhv4\" (UID: \"45621191-5080-4ee9-a08e-f7b2ee49fe41\") " pod="openshift-network-operator/iptables-alerter-sdhv4"
Nov 04 03:17:38.696586 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.696460    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d604b334-07f4-40f2-bfa9-4b26c301c84c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jvtr8\" (UID: \"d604b334-07f4-40f2-bfa9-4b26c301c84c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8"
Nov 04 03:17:38.696586 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.696480    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d604b334-07f4-40f2-bfa9-4b26c301c84c-socket-dir\") pod \"aws-ebs-csi-driver-node-jvtr8\" (UID: \"d604b334-07f4-40f2-bfa9-4b26c301c84c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8"
Nov 04 03:17:38.696586 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.696512    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-host-var-lib-cni-multus\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:17:38.696780 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.696600    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ff9f5afb-957c-4383-8fa9-698f7ada29b0-cni-binary-copy\") pod \"multus-additional-cni-plugins-67bjk\" (UID: \"ff9f5afb-957c-4383-8fa9-698f7ada29b0\") " pod="openshift-multus/multus-additional-cni-plugins-67bjk"
Nov 04 03:17:38.696780 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.696651    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c1b21c41-4b51-4da8-8920-c38cf49554bf-default-certificate\") pod \"router-default-7cfb798487-kbfzk\" (UID: \"c1b21c41-4b51-4da8-8920-c38cf49554bf\") " pod="openshift-ingress/router-default-7cfb798487-kbfzk"
Nov 04 03:17:38.696780 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.696721    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/029e7b7b-265e-459a-bc09-a90b5c179993-konnectivity-ca\") pod \"konnectivity-agent-m2vkr\" (UID: \"029e7b7b-265e-459a-bc09-a90b5c179993\") " pod="kube-system/konnectivity-agent-m2vkr"
Nov 04 03:17:38.696919 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.696836    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-cnibin\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:17:38.696919 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.696875    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fc612506-b699-44bc-a81b-06fee4e819fe-ca-trust-extracted\") pod \"image-registry-7cb9f7c89d-wnpll\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") " pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:17:38.697003 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.696916    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/45621191-5080-4ee9-a08e-f7b2ee49fe41-iptables-alerter-script\") pod \"iptables-alerter-sdhv4\" (UID: \"45621191-5080-4ee9-a08e-f7b2ee49fe41\") " pod="openshift-network-operator/iptables-alerter-sdhv4"
Nov 04 03:17:38.697003 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.696923    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3b178ce9-e140-4420-81f1-627de00b90c6-serving-cert\") pod \"insights-operator-7c4986d449-hsb49\" (UID: \"3b178ce9-e140-4420-81f1-627de00b90c6\") " pod="openshift-insights/insights-operator-7c4986d449-hsb49"
Nov 04 03:17:38.697003 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.696926    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c1b21c41-4b51-4da8-8920-c38cf49554bf-default-certificate\") pod \"router-default-7cfb798487-kbfzk\" (UID: \"c1b21c41-4b51-4da8-8920-c38cf49554bf\") " pod="openshift-ingress/router-default-7cfb798487-kbfzk"
Nov 04 03:17:38.697003 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.696950    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e487dae4-8ac9-4acd-a8ad-a0d149909475-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-4z8fp\" (UID: \"e487dae4-8ac9-4acd-a8ad-a0d149909475\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-4z8fp"
Nov 04 03:17:38.697003 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.696986    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-etc-systemd\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:17:38.697003 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.696991    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fc612506-b699-44bc-a81b-06fee4e819fe-ca-trust-extracted\") pod \"image-registry-7cb9f7c89d-wnpll\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") " pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:17:38.697241 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.697004    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-lib-modules\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:17:38.697241 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.697021    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kl5q\" (UniqueName: \"kubernetes.io/projected/6f627240-24ef-4aee-9810-51f422083ca3-kube-api-access-2kl5q\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:17:38.697241 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.697164    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e487dae4-8ac9-4acd-a8ad-a0d149909475-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-4z8fp\" (UID: \"e487dae4-8ac9-4acd-a8ad-a0d149909475\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-4z8fp"
Nov 04 03:17:38.697387 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.697256    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbrl5\" (UniqueName: \"kubernetes.io/projected/e87dded5-1e1b-45cb-ab2e-ae12550dfca7-kube-api-access-sbrl5\") pod \"cluster-monitoring-operator-884576c57-gmt9g\" (UID: \"e87dded5-1e1b-45cb-ab2e-ae12550dfca7\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmt9g"
Nov 04 03:17:38.697387 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.697290    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-host-run-netns\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.697387 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.697317    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsjms\" (UniqueName: \"kubernetes.io/projected/c1b21c41-4b51-4da8-8920-c38cf49554bf-kube-api-access-fsjms\") pod \"router-default-7cfb798487-kbfzk\" (UID: \"c1b21c41-4b51-4da8-8920-c38cf49554bf\") " pod="openshift-ingress/router-default-7cfb798487-kbfzk"
Nov 04 03:17:38.697387 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.697357    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjtnj\" (UniqueName: \"kubernetes.io/projected/f1f61b42-3b07-4892-90a7-2efa024e435b-kube-api-access-kjtnj\") pod \"image-registry-79d6dd87ff-n2jnc\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") " pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:17:38.697599 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.697456    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5krfn\" (UniqueName: \"kubernetes.io/projected/fc3d3c0e-2877-4264-93a7-be3756d2dc5c-kube-api-access-5krfn\") pod \"network-metrics-daemon-lngrr\" (UID: \"fc3d3c0e-2877-4264-93a7-be3756d2dc5c\") " pod="openshift-multus/network-metrics-daemon-lngrr"
Nov 04 03:17:38.697599 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.697493    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ff9f5afb-957c-4383-8fa9-698f7ada29b0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-67bjk\" (UID: \"ff9f5afb-957c-4383-8fa9-698f7ada29b0\") " pod="openshift-multus/multus-additional-cni-plugins-67bjk"
Nov 04 03:17:38.697599 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.697514    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x424d\" (UniqueName: \"kubernetes.io/projected/ff9f5afb-957c-4383-8fa9-698f7ada29b0-kube-api-access-x424d\") pod \"multus-additional-cni-plugins-67bjk\" (UID: \"ff9f5afb-957c-4383-8fa9-698f7ada29b0\") " pod="openshift-multus/multus-additional-cni-plugins-67bjk"
Nov 04 03:17:38.697599 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.697540    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3b178ce9-e140-4420-81f1-627de00b90c6-tmp\") pod \"insights-operator-7c4986d449-hsb49\" (UID: \"3b178ce9-e140-4420-81f1-627de00b90c6\") " pod="openshift-insights/insights-operator-7c4986d449-hsb49"
Nov 04 03:17:38.697599 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.697583    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6c8b9\" (UniqueName: \"kubernetes.io/projected/45621191-5080-4ee9-a08e-f7b2ee49fe41-kube-api-access-6c8b9\") pod \"iptables-alerter-sdhv4\" (UID: \"45621191-5080-4ee9-a08e-f7b2ee49fe41\") " pod="openshift-network-operator/iptables-alerter-sdhv4"
Nov 04 03:17:38.697823 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.697623    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/21a42913-5c4e-4057-aae5-2804038f16e4-env-overrides\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.697823 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.697654    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/21a42913-5c4e-4057-aae5-2804038f16e4-ovnkube-script-lib\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.697823 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.697673    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3b178ce9-e140-4420-81f1-627de00b90c6-tmp\") pod \"insights-operator-7c4986d449-hsb49\" (UID: \"3b178ce9-e140-4420-81f1-627de00b90c6\") " pod="openshift-insights/insights-operator-7c4986d449-hsb49"
Nov 04 03:17:38.697823 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.697693    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-etc-kubernetes\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:17:38.697823 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.697747    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/06698c23-887e-4ee5-9728-04098200c13d-metrics-tls\") pod \"dns-default-s4852\" (UID: \"06698c23-887e-4ee5-9728-04098200c13d\") " pod="openshift-dns/dns-default-s4852"
Nov 04 03:17:38.697823 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.697785    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mbsz\" (UniqueName: \"kubernetes.io/projected/6dedccdc-aea2-4664-8480-ca3c9359437d-kube-api-access-5mbsz\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-dln2k\" (UID: \"6dedccdc-aea2-4664-8480-ca3c9359437d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dln2k"
Nov 04 03:17:38.697823 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.697815    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4pj6\" (UniqueName: \"kubernetes.io/projected/0beb091e-5c2f-4136-93a2-771f9b861411-kube-api-access-m4pj6\") pod \"service-ca-operator-544d749c8c-js5xd\" (UID: \"0beb091e-5c2f-4136-93a2-771f9b861411\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-js5xd"
Nov 04 03:17:38.698081 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.697850    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-host-cni-netd\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.698081 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.697875    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/21a42913-5c4e-4057-aae5-2804038f16e4-ovn-node-metrics-cert\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.698081 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.697898    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/536a0057-a438-4ad9-a4cd-b26cb7f3c645-crio-socket\") pod \"insights-runtime-extractor-46btb\" (UID: \"536a0057-a438-4ad9-a4cd-b26cb7f3c645\") " pod="openshift-insights/insights-runtime-extractor-46btb"
Nov 04 03:17:38.698081 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.697921    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-etc-modprobe-d\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:17:38.698081 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.697949    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66hms\" (UniqueName: \"kubernetes.io/projected/e0c72601-2624-4632-9f15-c19fcc489aae-kube-api-access-66hms\") pod \"kas-connection-checker-685895c7c9-mdchx\" (UID: \"e0c72601-2624-4632-9f15-c19fcc489aae\") " pod="kube-system/kas-connection-checker-685895c7c9-mdchx"
Nov 04 03:17:38.698081 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.697995    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7713890e-763d-4a51-bffb-6cf8dd12fd18-host\") pod \"node-ca-n4b5s\" (UID: \"7713890e-763d-4a51-bffb-6cf8dd12fd18\") " pod="openshift-image-registry/node-ca-n4b5s"
Nov 04 03:17:38.698081 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.698032    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hp4bb\" (UniqueName: \"kubernetes.io/projected/7713890e-763d-4a51-bffb-6cf8dd12fd18-kube-api-access-hp4bb\") pod \"node-ca-n4b5s\" (UID: \"7713890e-763d-4a51-bffb-6cf8dd12fd18\") " pod="openshift-image-registry/node-ca-n4b5s"
Nov 04 03:17:38.698316 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.698094    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d604b334-07f4-40f2-bfa9-4b26c301c84c-sys-fs\") pod \"aws-ebs-csi-driver-node-jvtr8\" (UID: \"d604b334-07f4-40f2-bfa9-4b26c301c84c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8"
Nov 04 03:17:38.698316 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.698129    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6f627240-24ef-4aee-9810-51f422083ca3-cni-binary-copy\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:17:38.698316 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.698160    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fc612506-b699-44bc-a81b-06fee4e819fe-image-registry-private-configuration\") pod \"image-registry-7cb9f7c89d-wnpll\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") " pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:17:38.698316 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.698177    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/068619f8-7fbb-4b22-ac7d-a1f144e084c0-serving-cert\") pod \"console-operator-7cd56b8dbc-p9989\" (UID: \"068619f8-7fbb-4b22-ac7d-a1f144e084c0\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-p9989"
Nov 04 03:17:38.698316 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.698197    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f1f61b42-3b07-4892-90a7-2efa024e435b-bound-sa-token\") pod \"image-registry-79d6dd87ff-n2jnc\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") " pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:17:38.698316 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.698264    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/536a0057-a438-4ad9-a4cd-b26cb7f3c645-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-46btb\" (UID: \"536a0057-a438-4ad9-a4cd-b26cb7f3c645\") " pod="openshift-insights/insights-runtime-extractor-46btb"
Nov 04 03:17:38.698316 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.698300    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jltvv\" (UniqueName: \"kubernetes.io/projected/04b955ef-528b-4573-b18f-0957658f79e4-kube-api-access-jltvv\") pod \"network-check-target-hjssc\" (UID: \"04b955ef-528b-4573-b18f-0957658f79e4\") " pod="openshift-network-diagnostics/network-check-target-hjssc"
Nov 04 03:17:38.698670 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.698327    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f1f61b42-3b07-4892-90a7-2efa024e435b-ca-trust-extracted\") pod \"image-registry-79d6dd87ff-n2jnc\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") " pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:17:38.698670 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.698357    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6dedccdc-aea2-4664-8480-ca3c9359437d-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-dln2k\" (UID: \"6dedccdc-aea2-4664-8480-ca3c9359437d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dln2k"
Nov 04 03:17:38.698670 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.698396    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7713890e-763d-4a51-bffb-6cf8dd12fd18-serviceca\") pod \"node-ca-n4b5s\" (UID: \"7713890e-763d-4a51-bffb-6cf8dd12fd18\") " pod="openshift-image-registry/node-ca-n4b5s"
Nov 04 03:17:38.698670 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.698435    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/068619f8-7fbb-4b22-ac7d-a1f144e084c0-serving-cert\") pod \"console-operator-7cd56b8dbc-p9989\" (UID: \"068619f8-7fbb-4b22-ac7d-a1f144e084c0\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-p9989"
Nov 04 03:17:38.698670 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.698444    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f1f61b42-3b07-4892-90a7-2efa024e435b-ca-trust-extracted\") pod \"image-registry-79d6dd87ff-n2jnc\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") " pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:17:38.698670 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.698437    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1b21c41-4b51-4da8-8920-c38cf49554bf-service-ca-bundle\") pod \"router-default-7cfb798487-kbfzk\" (UID: \"c1b21c41-4b51-4da8-8920-c38cf49554bf\") " pod="openshift-ingress/router-default-7cfb798487-kbfzk"
Nov 04 03:17:38.698670 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.698482    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f1f61b42-3b07-4892-90a7-2efa024e435b-trusted-ca\") pod \"image-registry-79d6dd87ff-n2jnc\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") " pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:17:38.698670 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.698511    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/06698c23-887e-4ee5-9728-04098200c13d-config-volume\") pod \"dns-default-s4852\" (UID: \"06698c23-887e-4ee5-9728-04098200c13d\") " pod="openshift-dns/dns-default-s4852"
Nov 04 03:17:38.698670 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.698541    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/e487dae4-8ac9-4acd-a8ad-a0d149909475-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-4z8fp\" (UID: \"e487dae4-8ac9-4acd-a8ad-a0d149909475\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-4z8fp"
Nov 04 03:17:38.698670 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.698553    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6dedccdc-aea2-4664-8480-ca3c9359437d-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-dln2k\" (UID: \"6dedccdc-aea2-4664-8480-ca3c9359437d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dln2k"
Nov 04 03:17:38.698670 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.698581    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-hostroot\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:17:38.698670 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.698613    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1b21c41-4b51-4da8-8920-c38cf49554bf-service-ca-bundle\") pod \"router-default-7cfb798487-kbfzk\" (UID: \"c1b21c41-4b51-4da8-8920-c38cf49554bf\") " pod="openshift-ingress/router-default-7cfb798487-kbfzk"
Nov 04 03:17:38.698670 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.698622    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ldp7\" (UniqueName: \"kubernetes.io/projected/a51f26a7-0ec8-4c56-881a-98c6d625478b-kube-api-access-2ldp7\") pod \"kas-connection-checker-685895c7c9-xpmwt\" (UID: \"a51f26a7-0ec8-4c56-881a-98c6d625478b\") " pod="kube-system/kas-connection-checker-685895c7c9-xpmwt"
Nov 04 03:17:38.698670 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.698656    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f1f61b42-3b07-4892-90a7-2efa024e435b-image-registry-private-configuration\") pod \"image-registry-79d6dd87ff-n2jnc\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") " pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:17:38.698670 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.698676    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fhxt\" (UniqueName: \"kubernetes.io/projected/536a0057-a438-4ad9-a4cd-b26cb7f3c645-kube-api-access-7fhxt\") pod \"insights-runtime-extractor-46btb\" (UID: \"536a0057-a438-4ad9-a4cd-b26cb7f3c645\") " pod="openshift-insights/insights-runtime-extractor-46btb"
Nov 04 03:17:38.699368 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.698693    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-host-kubelet\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.699368 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.698719    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-host\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:17:38.699368 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.698744    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/e487dae4-8ac9-4acd-a8ad-a0d149909475-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-4z8fp\" (UID: \"e487dae4-8ac9-4acd-a8ad-a0d149909475\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-4z8fp"
Nov 04 03:17:38.699368 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.698789    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-multus-conf-dir\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:17:38.699368 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.698820    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/068619f8-7fbb-4b22-ac7d-a1f144e084c0-trusted-ca\") pod \"console-operator-7cd56b8dbc-p9989\" (UID: \"068619f8-7fbb-4b22-ac7d-a1f144e084c0\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-p9989"
Nov 04 03:17:38.699368 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.698857    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b178ce9-e140-4420-81f1-627de00b90c6-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-hsb49\" (UID: \"3b178ce9-e140-4420-81f1-627de00b90c6\") " pod="openshift-insights/insights-operator-7c4986d449-hsb49"
Nov 04 03:17:38.699368 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.698881    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b178ce9-e140-4420-81f1-627de00b90c6-service-ca-bundle\") pod \"insights-operator-7c4986d449-hsb49\" (UID: \"3b178ce9-e140-4420-81f1-627de00b90c6\") " pod="openshift-insights/insights-operator-7c4986d449-hsb49"
Nov 04 03:17:38.699368 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.698900    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f1f61b42-3b07-4892-90a7-2efa024e435b-registry-certificates\") pod \"image-registry-79d6dd87ff-n2jnc\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") " pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:17:38.699368 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.699008    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-etc-sysctl-d\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:17:38.699368 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.699039    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-etc-sysconfig\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:17:38.699368 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.699074    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ff9f5afb-957c-4383-8fa9-698f7ada29b0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-67bjk\" (UID: \"ff9f5afb-957c-4383-8fa9-698f7ada29b0\") " pod="openshift-multus/multus-additional-cni-plugins-67bjk"
Nov 04 03:17:38.699368 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.699125    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/068619f8-7fbb-4b22-ac7d-a1f144e084c0-trusted-ca\") pod \"console-operator-7cd56b8dbc-p9989\" (UID: \"068619f8-7fbb-4b22-ac7d-a1f144e084c0\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-p9989"
Nov 04 03:17:38.699368 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.699129    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fc612506-b699-44bc-a81b-06fee4e819fe-registry-certificates\") pod \"image-registry-7cb9f7c89d-wnpll\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") " pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:17:38.699368 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.699183    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9trx7\" (UniqueName: \"kubernetes.io/projected/21a42913-5c4e-4057-aae5-2804038f16e4-kube-api-access-9trx7\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.699368 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.699184    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b178ce9-e140-4420-81f1-627de00b90c6-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-hsb49\" (UID: \"3b178ce9-e140-4420-81f1-627de00b90c6\") " pod="openshift-insights/insights-operator-7c4986d449-hsb49"
Nov 04 03:17:38.699368 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.699223    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f1f61b42-3b07-4892-90a7-2efa024e435b-installation-pull-secrets\") pod \"image-registry-79d6dd87ff-n2jnc\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") " pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:17:38.699368 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.699293    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-multus-cni-dir\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:17:38.699970 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.699378    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6dedccdc-aea2-4664-8480-ca3c9359437d-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-dln2k\" (UID: \"6dedccdc-aea2-4664-8480-ca3c9359437d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dln2k"
Nov 04 03:17:38.699970 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.699395    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b178ce9-e140-4420-81f1-627de00b90c6-service-ca-bundle\") pod \"insights-operator-7c4986d449-hsb49\" (UID: \"3b178ce9-e140-4420-81f1-627de00b90c6\") " pod="openshift-insights/insights-operator-7c4986d449-hsb49"
Nov 04 03:17:38.699970 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.699458    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d731646d-23ab-42d7-ae48-974929b3d2dc-nginx-conf\") pod \"networking-console-plugin-86c478bb88-tlhng\" (UID: \"d731646d-23ab-42d7-ae48-974929b3d2dc\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-tlhng"
Nov 04 03:17:38.699970 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.699486    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4sqp\" (UniqueName: \"kubernetes.io/projected/06698c23-887e-4ee5-9728-04098200c13d-kube-api-access-v4sqp\") pod \"dns-default-s4852\" (UID: \"06698c23-887e-4ee5-9728-04098200c13d\") " pod="openshift-dns/dns-default-s4852"
Nov 04 03:17:38.699970 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.699549    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-run-systemd\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.699970 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.699582    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d10e9268-df07-428b-ae25-528d9fe075d2-dbus\") pod \"global-pull-secret-syncer-tkkhk\" (UID: \"d10e9268-df07-428b-ae25-528d9fe075d2\") " pod="kube-system/global-pull-secret-syncer-tkkhk"
Nov 04 03:17:38.699970 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.699603    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d731646d-23ab-42d7-ae48-974929b3d2dc-nginx-conf\") pod \"networking-console-plugin-86c478bb88-tlhng\" (UID: \"d731646d-23ab-42d7-ae48-974929b3d2dc\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-tlhng"
Nov 04 03:17:38.699970 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.699610    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-etc-sysctl-conf\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:17:38.699970 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.699661    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-host-run-k8s-cni-cncf-io\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:17:38.699970 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.699700    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d731646d-23ab-42d7-ae48-974929b3d2dc-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-tlhng\" (UID: \"d731646d-23ab-42d7-ae48-974929b3d2dc\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-tlhng"
Nov 04 03:17:38.699970 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.699732    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-run-ovn\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.699970 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.699740    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6dedccdc-aea2-4664-8480-ca3c9359437d-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-dln2k\" (UID: \"6dedccdc-aea2-4664-8480-ca3c9359437d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dln2k"
Nov 04 03:17:38.699970 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.699765    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qm4f8\" (UniqueName: \"kubernetes.io/projected/a683bd45-33c3-43fb-9530-292d7fea6970-kube-api-access-qm4f8\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:17:38.699970 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.699790    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-host-run-multus-certs\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:17:38.699970 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.699806    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5h469\" (UniqueName: \"kubernetes.io/projected/55c4af4c-188b-495b-9052-5700a1f60602-kube-api-access-5h469\") pod \"ingress-canary-qbskg\" (UID: \"55c4af4c-188b-495b-9052-5700a1f60602\") " pod="openshift-ingress-canary/ingress-canary-qbskg"
Nov 04 03:17:38.699970 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.699832    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d604b334-07f4-40f2-bfa9-4b26c301c84c-registration-dir\") pod \"aws-ebs-csi-driver-node-jvtr8\" (UID: \"d604b334-07f4-40f2-bfa9-4b26c301c84c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8"
Nov 04 03:17:38.699970 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.699855    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-etc-kubernetes\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:17:38.699970 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.699876    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-597ml\" (UniqueName: \"kubernetes.io/projected/10cd06d4-ef1a-4ecf-a72c-e6d0084f3032-kube-api-access-597ml\") pod \"kas-connection-checker-685895c7c9-n6r7s\" (UID: \"10cd06d4-ef1a-4ecf-a72c-e6d0084f3032\") " pod="kube-system/kas-connection-checker-685895c7c9-n6r7s"
Nov 04 03:17:38.699970 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.699893    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-systemd-units\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.699970 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.699933    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a683bd45-33c3-43fb-9530-292d7fea6970-etc-tuned\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:17:38.699970 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.699940    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d731646d-23ab-42d7-ae48-974929b3d2dc-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-tlhng\" (UID: \"d731646d-23ab-42d7-ae48-974929b3d2dc\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-tlhng"
Nov 04 03:17:38.699970 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.699956    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ff9f5afb-957c-4383-8fa9-698f7ada29b0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-67bjk\" (UID: \"ff9f5afb-957c-4383-8fa9-698f7ada29b0\") " pod="openshift-multus/multus-additional-cni-plugins-67bjk"
Nov 04 03:17:38.699970 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.699980    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghplp\" (UniqueName: \"kubernetes.io/projected/3b178ce9-e140-4420-81f1-627de00b90c6-kube-api-access-ghplp\") pod \"insights-operator-7c4986d449-hsb49\" (UID: \"3b178ce9-e140-4420-81f1-627de00b90c6\") " pod="openshift-insights/insights-operator-7c4986d449-hsb49"
Nov 04 03:17:38.700789 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.700019    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a683bd45-33c3-43fb-9530-292d7fea6970-etc-tuned\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:17:38.700789 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.700031    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-var-lib-openvswitch\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.700789 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.700073    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5cdl\" (UniqueName: \"kubernetes.io/projected/d604b334-07f4-40f2-bfa9-4b26c301c84c-kube-api-access-n5cdl\") pod \"aws-ebs-csi-driver-node-jvtr8\" (UID: \"d604b334-07f4-40f2-bfa9-4b26c301c84c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8"
Nov 04 03:17:38.700789 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.700149    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-sys\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:17:38.700789 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.700170    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-host-run-netns\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:17:38.700789 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.700188    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fc612506-b699-44bc-a81b-06fee4e819fe-registry-tls\") pod \"image-registry-7cb9f7c89d-wnpll\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") " pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:17:38.700789 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.700205    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0beb091e-5c2f-4136-93a2-771f9b861411-config\") pod \"service-ca-operator-544d749c8c-js5xd\" (UID: \"0beb091e-5c2f-4136-93a2-771f9b861411\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-js5xd"
Nov 04 03:17:38.700789 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.700310    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d10e9268-df07-428b-ae25-528d9fe075d2-original-pull-secret\") pod \"global-pull-secret-syncer-tkkhk\" (UID: \"d10e9268-df07-428b-ae25-528d9fe075d2\") " pod="kube-system/global-pull-secret-syncer-tkkhk"
Nov 04 03:17:38.700789 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.700327    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-run\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:17:38.700789 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.700342    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-host-var-lib-cni-bin\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:17:38.700789 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.700362    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fc612506-b699-44bc-a81b-06fee4e819fe-trusted-ca\") pod \"image-registry-7cb9f7c89d-wnpll\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") " pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:17:38.700789 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.700417    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfc98\" (UniqueName: \"kubernetes.io/projected/fc612506-b699-44bc-a81b-06fee4e819fe-kube-api-access-bfc98\") pod \"image-registry-7cb9f7c89d-wnpll\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") " pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:17:38.700789 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.700425    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0beb091e-5c2f-4136-93a2-771f9b861411-config\") pod \"service-ca-operator-544d749c8c-js5xd\" (UID: \"0beb091e-5c2f-4136-93a2-771f9b861411\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-js5xd"
Nov 04 03:17:38.700789 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.700440    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/21a42913-5c4e-4057-aae5-2804038f16e4-ovnkube-config\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.700789 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.700474    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d10e9268-df07-428b-ae25-528d9fe075d2-kubelet-config\") pod \"global-pull-secret-syncer-tkkhk\" (UID: \"d10e9268-df07-428b-ae25-528d9fe075d2\") " pod="kube-system/global-pull-secret-syncer-tkkhk"
Nov 04 03:17:38.700789 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.700495    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-host-var-lib-kubelet\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:17:38.700789 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.700533    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3b178ce9-e140-4420-81f1-627de00b90c6-snapshots\") pod \"insights-operator-7c4986d449-hsb49\" (UID: \"3b178ce9-e140-4420-81f1-627de00b90c6\") " pod="openshift-insights/insights-operator-7c4986d449-hsb49"
Nov 04 03:17:38.700789 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.700562    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e87dded5-1e1b-45cb-ab2e-ae12550dfca7-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-gmt9g\" (UID: \"e87dded5-1e1b-45cb-ab2e-ae12550dfca7\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmt9g"
Nov 04 03:17:38.700789 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.700600    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-host-run-ovn-kubernetes\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.700789 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.700629    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c1b21c41-4b51-4da8-8920-c38cf49554bf-stats-auth\") pod \"router-default-7cfb798487-kbfzk\" (UID: \"c1b21c41-4b51-4da8-8920-c38cf49554bf\") " pod="openshift-ingress/router-default-7cfb798487-kbfzk"
Nov 04 03:17:38.700789 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.700678    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d604b334-07f4-40f2-bfa9-4b26c301c84c-device-dir\") pod \"aws-ebs-csi-driver-node-jvtr8\" (UID: \"d604b334-07f4-40f2-bfa9-4b26c301c84c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8"
Nov 04 03:17:38.700789 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.700688    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3b178ce9-e140-4420-81f1-627de00b90c6-snapshots\") pod \"insights-operator-7c4986d449-hsb49\" (UID: \"3b178ce9-e140-4420-81f1-627de00b90c6\") " pod="openshift-insights/insights-operator-7c4986d449-hsb49"
Nov 04 03:17:38.700789 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.700720    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06698c23-887e-4ee5-9728-04098200c13d-tmp-dir\") pod \"dns-default-s4852\" (UID: \"06698c23-887e-4ee5-9728-04098200c13d\") " pod="openshift-dns/dns-default-s4852"
Nov 04 03:17:38.700789 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.700749    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ff9f5afb-957c-4383-8fa9-698f7ada29b0-system-cni-dir\") pod \"multus-additional-cni-plugins-67bjk\" (UID: \"ff9f5afb-957c-4383-8fa9-698f7ada29b0\") " pod="openshift-multus/multus-additional-cni-plugins-67bjk"
Nov 04 03:17:38.700789 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.700758    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e87dded5-1e1b-45cb-ab2e-ae12550dfca7-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-gmt9g\" (UID: \"e87dded5-1e1b-45cb-ab2e-ae12550dfca7\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmt9g"
Nov 04 03:17:38.700789 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.700769    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ff9f5afb-957c-4383-8fa9-698f7ada29b0-cnibin\") pod \"multus-additional-cni-plugins-67bjk\" (UID: \"ff9f5afb-957c-4383-8fa9-698f7ada29b0\") " pod="openshift-multus/multus-additional-cni-plugins-67bjk"
Nov 04 03:17:38.700789 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.700788    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c1b21c41-4b51-4da8-8920-c38cf49554bf-stats-auth\") pod \"router-default-7cfb798487-kbfzk\" (UID: \"c1b21c41-4b51-4da8-8920-c38cf49554bf\") " pod="openshift-ingress/router-default-7cfb798487-kbfzk"
Nov 04 03:17:38.701828 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.700830    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06698c23-887e-4ee5-9728-04098200c13d-tmp-dir\") pod \"dns-default-s4852\" (UID: \"06698c23-887e-4ee5-9728-04098200c13d\") " pod="openshift-dns/dns-default-s4852"
Nov 04 03:17:38.701828 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.700864    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fc612506-b699-44bc-a81b-06fee4e819fe-installation-pull-secrets\") pod \"image-registry-7cb9f7c89d-wnpll\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") " pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:17:38.701828 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.700893    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/55c4af4c-188b-495b-9052-5700a1f60602-cert\") pod \"ingress-canary-qbskg\" (UID: \"55c4af4c-188b-495b-9052-5700a1f60602\") " pod="openshift-ingress-canary/ingress-canary-qbskg"
Nov 04 03:17:38.701828 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.700961    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/536a0057-a438-4ad9-a4cd-b26cb7f3c645-data-volume\") pod \"insights-runtime-extractor-46btb\" (UID: \"536a0057-a438-4ad9-a4cd-b26cb7f3c645\") " pod="openshift-insights/insights-runtime-extractor-46btb"
Nov 04 03:17:38.701828 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.701051    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d604b334-07f4-40f2-bfa9-4b26c301c84c-etc-selinux\") pod \"aws-ebs-csi-driver-node-jvtr8\" (UID: \"d604b334-07f4-40f2-bfa9-4b26c301c84c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8"
Nov 04 03:17:38.701828 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.701069    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/536a0057-a438-4ad9-a4cd-b26cb7f3c645-data-volume\") pod \"insights-runtime-extractor-46btb\" (UID: \"536a0057-a438-4ad9-a4cd-b26cb7f3c645\") " pod="openshift-insights/insights-runtime-extractor-46btb"
Nov 04 03:17:38.701828 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.701102    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a683bd45-33c3-43fb-9530-292d7fea6970-tmp\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:17:38.701828 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.701121    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7cm5\" (UniqueName: \"kubernetes.io/projected/aab0bd9b-cb90-4d9b-9f46-52c6c87a34cf-kube-api-access-b7cm5\") pod \"node-resolver-6pwhj\" (UID: \"aab0bd9b-cb90-4d9b-9f46-52c6c87a34cf\") " pod="openshift-dns/node-resolver-6pwhj"
Nov 04 03:17:38.701828 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.701142    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a683bd45-33c3-43fb-9530-292d7fea6970-tmp\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:17:38.701828 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.701141    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0beb091e-5c2f-4136-93a2-771f9b861411-serving-cert\") pod \"service-ca-operator-544d749c8c-js5xd\" (UID: \"0beb091e-5c2f-4136-93a2-771f9b861411\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-js5xd"
Nov 04 03:17:38.701828 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.701202    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e87dded5-1e1b-45cb-ab2e-ae12550dfca7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-gmt9g\" (UID: \"e87dded5-1e1b-45cb-ab2e-ae12550dfca7\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmt9g"
Nov 04 03:17:38.701828 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.701232    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/aab0bd9b-cb90-4d9b-9f46-52c6c87a34cf-hosts-file\") pod \"node-resolver-6pwhj\" (UID: \"aab0bd9b-cb90-4d9b-9f46-52c6c87a34cf\") " pod="openshift-dns/node-resolver-6pwhj"
Nov 04 03:17:38.701828 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.701254    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-system-cni-dir\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:17:38.701828 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.701282    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fc612506-b699-44bc-a81b-06fee4e819fe-bound-sa-token\") pod \"image-registry-7cb9f7c89d-wnpll\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") " pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:17:38.701828 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.701307    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/068619f8-7fbb-4b22-ac7d-a1f144e084c0-config\") pod \"console-operator-7cd56b8dbc-p9989\" (UID: \"068619f8-7fbb-4b22-ac7d-a1f144e084c0\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-p9989"
Nov 04 03:17:38.701828 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.701361    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcm4d\" (UniqueName: \"kubernetes.io/projected/068619f8-7fbb-4b22-ac7d-a1f144e084c0-kube-api-access-mcm4d\") pod \"console-operator-7cd56b8dbc-p9989\" (UID: \"068619f8-7fbb-4b22-ac7d-a1f144e084c0\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-p9989"
Nov 04 03:17:38.701828 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.701493    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e87dded5-1e1b-45cb-ab2e-ae12550dfca7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-gmt9g\" (UID: \"e87dded5-1e1b-45cb-ab2e-ae12550dfca7\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmt9g"
Nov 04 03:17:38.701828 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.701497    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0beb091e-5c2f-4136-93a2-771f9b861411-serving-cert\") pod \"service-ca-operator-544d749c8c-js5xd\" (UID: \"0beb091e-5c2f-4136-93a2-771f9b861411\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-js5xd"
Nov 04 03:17:38.701828 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.701514    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/068619f8-7fbb-4b22-ac7d-a1f144e084c0-config\") pod \"console-operator-7cd56b8dbc-p9989\" (UID: \"068619f8-7fbb-4b22-ac7d-a1f144e084c0\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-p9989"
Nov 04 03:17:38.737238 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.737142    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 04 03:17:38.737238 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.737168    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 04 03:17:38.737238 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.737157    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 04 03:17:38.737238 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.737211    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 04 03:17:38.737575 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.737144    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 04 03:17:38.737575 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.737323    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 04 03:17:38.737575 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.737421    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 04 03:17:38.737575 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.737449    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 04 03:17:38.737575 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.737467    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 04 03:17:38.737575 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.737556    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 04 03:17:38.737926 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.737584    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 04 03:17:38.737926 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.737657    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 04 03:17:38.737926 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.737681    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 04 03:17:38.737926 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.737686    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 04 03:17:38.737926 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.737745    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 04 03:17:38.737926 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.737850    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 04 03:17:38.737926 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.737861    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 04 03:17:38.737926 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.737874    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 04 03:17:38.738247 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.737992    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 04 03:17:38.738247 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.737997    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 04 03:17:38.738247 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.738006    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/21a42913-5c4e-4057-aae5-2804038f16e4-env-overrides\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.738247 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.738012    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 04 03:17:38.738247 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.738053    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 04 03:17:38.738247 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.738085    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 04 03:17:38.738247 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.738093    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 04 03:17:38.738566 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.738308    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 04 03:17:38.738566 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.738378    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 04 03:17:38.738566 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.738476    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 04 03:17:38.738566 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.738511    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6f627240-24ef-4aee-9810-51f422083ca3-cni-binary-copy\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:17:38.738566 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.738537    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 04 03:17:38.738861 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.738603    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Nov 04 03:17:38.738861 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.738726    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 04 03:17:38.738861 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.738791    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/536a0057-a438-4ad9-a4cd-b26cb7f3c645-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-46btb\" (UID: \"536a0057-a438-4ad9-a4cd-b26cb7f3c645\") " pod="openshift-insights/insights-runtime-extractor-46btb"
Nov 04 03:17:38.739274 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.738867    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 04 03:17:38.739274 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.738903    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-mnqmt\""
Nov 04 03:17:38.739274 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.738916    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-fbcnm\""
Nov 04 03:17:38.739274 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.738936    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 04 03:17:38.739274 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.738943    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f1f61b42-3b07-4892-90a7-2efa024e435b-image-registry-private-configuration\") pod \"image-registry-79d6dd87ff-n2jnc\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") " pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:17:38.739274 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.738967    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-x6xxb\""
Nov 04 03:17:38.739274 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.739008    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-56wj9\""
Nov 04 03:17:38.739274 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.739029    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-zsv6g\""
Nov 04 03:17:38.739274 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.739026    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7713890e-763d-4a51-bffb-6cf8dd12fd18-serviceca\") pod \"node-ca-n4b5s\" (UID: \"7713890e-763d-4a51-bffb-6cf8dd12fd18\") " pod="openshift-image-registry/node-ca-n4b5s"
Nov 04 03:17:38.739274 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.739041    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-gl4fp\""
Nov 04 03:17:38.739274 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.739087    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-mhdb8\""
Nov 04 03:17:38.739274 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.739100    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-z2nst\""
Nov 04 03:17:38.739274 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.739109    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 04 03:17:38.739274 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.739089    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Nov 04 03:17:38.739274 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.739128    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-4vrb5\""
Nov 04 03:17:38.739274 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.739135    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 04 03:17:38.739274 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.739140    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jw2mq\""
Nov 04 03:17:38.739274 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.739133    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vjndb\""
Nov 04 03:17:38.739274 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.739179    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f1f61b42-3b07-4892-90a7-2efa024e435b-registry-certificates\") pod \"image-registry-79d6dd87ff-n2jnc\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") " pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:17:38.739274 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.739207    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/06698c23-887e-4ee5-9728-04098200c13d-config-volume\") pod \"dns-default-s4852\" (UID: \"06698c23-887e-4ee5-9728-04098200c13d\") " pod="openshift-dns/dns-default-s4852"
Nov 04 03:17:38.740020 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.739342    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-h66cx\""
Nov 04 03:17:38.740020 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.739352    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-569bn\""
Nov 04 03:17:38.740020 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.739370    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xwzp4\""
Nov 04 03:17:38.740020 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.739391    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-vpndz\""
Nov 04 03:17:38.740020 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.739421    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Nov 04 03:17:38.740020 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.739465    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-9cxzk\""
Nov 04 03:17:38.740020 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.739488    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 04 03:17:38.740020 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.739536    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f1f61b42-3b07-4892-90a7-2efa024e435b-installation-pull-secrets\") pod \"image-registry-79d6dd87ff-n2jnc\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") " pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:17:38.740020 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.739581    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 04 03:17:38.740020 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.739632    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 04 03:17:38.740020 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.739763    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fc612506-b699-44bc-a81b-06fee4e819fe-registry-certificates\") pod \"image-registry-7cb9f7c89d-wnpll\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") " pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:17:38.740322 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.740241    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Nov 04 03:17:38.740375 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.740358    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ff9f5afb-957c-4383-8fa9-698f7ada29b0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-67bjk\" (UID: \"ff9f5afb-957c-4383-8fa9-698f7ada29b0\") " pod="openshift-multus/multus-additional-cni-plugins-67bjk"
Nov 04 03:17:38.740514 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.740497    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fc612506-b699-44bc-a81b-06fee4e819fe-registry-tls\") pod \"image-registry-7cb9f7c89d-wnpll\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") " pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:17:38.740663 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.740650    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/21a42913-5c4e-4057-aae5-2804038f16e4-ovnkube-config\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.740745 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.740731    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fc612506-b699-44bc-a81b-06fee4e819fe-trusted-ca\") pod \"image-registry-7cb9f7c89d-wnpll\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") " pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:17:38.740969 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.740954    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d10e9268-df07-428b-ae25-528d9fe075d2-original-pull-secret\") pod \"global-pull-secret-syncer-tkkhk\" (UID: \"d10e9268-df07-428b-ae25-528d9fe075d2\") " pod="kube-system/global-pull-secret-syncer-tkkhk"
Nov 04 03:17:38.741080 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.741064    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fc612506-b699-44bc-a81b-06fee4e819fe-installation-pull-secrets\") pod \"image-registry-7cb9f7c89d-wnpll\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") " pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:17:38.741202 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.741186    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/55c4af4c-188b-495b-9052-5700a1f60602-cert\") pod \"ingress-canary-qbskg\" (UID: \"55c4af4c-188b-495b-9052-5700a1f60602\") " pod="openshift-ingress-canary/ingress-canary-qbskg"
Nov 04 03:17:38.746374 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.746359    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/029e7b7b-265e-459a-bc09-a90b5c179993-agent-certs\") pod \"konnectivity-agent-m2vkr\" (UID: \"029e7b7b-265e-459a-bc09-a90b5c179993\") " pod="kube-system/konnectivity-agent-m2vkr"
Nov 04 03:17:38.746525 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.746507    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6f627240-24ef-4aee-9810-51f422083ca3-multus-daemon-config\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:17:38.746647 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.746635    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f1f61b42-3b07-4892-90a7-2efa024e435b-registry-tls\") pod \"image-registry-79d6dd87ff-n2jnc\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") " pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:17:38.746765 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.746753    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc3d3c0e-2877-4264-93a7-be3756d2dc5c-metrics-certs\") pod \"network-metrics-daemon-lngrr\" (UID: \"fc3d3c0e-2877-4264-93a7-be3756d2dc5c\") " pod="openshift-multus/network-metrics-daemon-lngrr"
Nov 04 03:17:38.746850 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.746835    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ff9f5afb-957c-4383-8fa9-698f7ada29b0-cni-binary-copy\") pod \"multus-additional-cni-plugins-67bjk\" (UID: \"ff9f5afb-957c-4383-8fa9-698f7ada29b0\") " pod="openshift-multus/multus-additional-cni-plugins-67bjk"
Nov 04 03:17:38.747958 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.747941    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/06698c23-887e-4ee5-9728-04098200c13d-metrics-tls\") pod \"dns-default-s4852\" (UID: \"06698c23-887e-4ee5-9728-04098200c13d\") " pod="openshift-dns/dns-default-s4852"
Nov 04 03:17:38.748029 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.747943    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ff9f5afb-957c-4383-8fa9-698f7ada29b0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-67bjk\" (UID: \"ff9f5afb-957c-4383-8fa9-698f7ada29b0\") " pod="openshift-multus/multus-additional-cni-plugins-67bjk"
Nov 04 03:17:38.748029 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.747965    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/029e7b7b-265e-459a-bc09-a90b5c179993-konnectivity-ca\") pod \"konnectivity-agent-m2vkr\" (UID: \"029e7b7b-265e-459a-bc09-a90b5c179993\") " pod="kube-system/konnectivity-agent-m2vkr"
Nov 04 03:17:38.748029 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.747986    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/45621191-5080-4ee9-a08e-f7b2ee49fe41-iptables-alerter-script\") pod \"iptables-alerter-sdhv4\" (UID: \"45621191-5080-4ee9-a08e-f7b2ee49fe41\") " pod="openshift-network-operator/iptables-alerter-sdhv4"
Nov 04 03:17:38.748179 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.748140    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/21a42913-5c4e-4057-aae5-2804038f16e4-ovnkube-script-lib\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.748228 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.748204    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/21a42913-5c4e-4057-aae5-2804038f16e4-ovn-node-metrics-cert\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.749263 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.749245    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fc612506-b699-44bc-a81b-06fee4e819fe-image-registry-private-configuration\") pod \"image-registry-7cb9f7c89d-wnpll\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") " pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:17:38.749510 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.749490    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f1f61b42-3b07-4892-90a7-2efa024e435b-trusted-ca\") pod \"image-registry-79d6dd87ff-n2jnc\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") " pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:17:38.751933 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.751915    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnrbg\" (UniqueName: \"kubernetes.io/projected/cc469246-2293-4301-9e46-bf9d609be964-kube-api-access-cnrbg\") pod \"network-check-source-6f78d59f99-7lkrx\" (UID: \"cc469246-2293-4301-9e46-bf9d609be964\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-7lkrx"
Nov 04 03:17:38.752361 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.752341    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlxxz\" (UniqueName: \"kubernetes.io/projected/e487dae4-8ac9-4acd-a8ad-a0d149909475-kube-api-access-vlxxz\") pod \"cluster-samples-operator-6ddb6c8b94-4z8fp\" (UID: \"e487dae4-8ac9-4acd-a8ad-a0d149909475\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-4z8fp"
Nov 04 03:17:38.752451 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.752427    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8g8qf\" (UniqueName: \"kubernetes.io/projected/e451b329-bd05-44f5-9544-8283f76df9d6-kube-api-access-8g8qf\") pod \"volume-data-source-validator-58c9f5f46d-6c2jp\" (UID: \"e451b329-bd05-44f5-9544-8283f76df9d6\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-6c2jp"
Nov 04 03:17:38.753060 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.753045    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kl5q\" (UniqueName: \"kubernetes.io/projected/6f627240-24ef-4aee-9810-51f422083ca3-kube-api-access-2kl5q\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:17:38.753464 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.753448    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsjms\" (UniqueName: \"kubernetes.io/projected/c1b21c41-4b51-4da8-8920-c38cf49554bf-kube-api-access-fsjms\") pod \"router-default-7cfb798487-kbfzk\" (UID: \"c1b21c41-4b51-4da8-8920-c38cf49554bf\") " pod="openshift-ingress/router-default-7cfb798487-kbfzk"
Nov 04 03:17:38.757199 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.757175    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4pj6\" (UniqueName: \"kubernetes.io/projected/0beb091e-5c2f-4136-93a2-771f9b861411-kube-api-access-m4pj6\") pod \"service-ca-operator-544d749c8c-js5xd\" (UID: \"0beb091e-5c2f-4136-93a2-771f9b861411\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-js5xd"
Nov 04 03:17:38.757283 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.757260    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mbsz\" (UniqueName: \"kubernetes.io/projected/6dedccdc-aea2-4664-8480-ca3c9359437d-kube-api-access-5mbsz\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-dln2k\" (UID: \"6dedccdc-aea2-4664-8480-ca3c9359437d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dln2k"
Nov 04 03:17:38.757359 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.757339    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x424d\" (UniqueName: \"kubernetes.io/projected/ff9f5afb-957c-4383-8fa9-698f7ada29b0-kube-api-access-x424d\") pod \"multus-additional-cni-plugins-67bjk\" (UID: \"ff9f5afb-957c-4383-8fa9-698f7ada29b0\") " pod="openshift-multus/multus-additional-cni-plugins-67bjk"
Nov 04 03:17:38.757445 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.757373    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjtnj\" (UniqueName: \"kubernetes.io/projected/f1f61b42-3b07-4892-90a7-2efa024e435b-kube-api-access-kjtnj\") pod \"image-registry-79d6dd87ff-n2jnc\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") " pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:17:38.757902 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.757886    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6c8b9\" (UniqueName: \"kubernetes.io/projected/45621191-5080-4ee9-a08e-f7b2ee49fe41-kube-api-access-6c8b9\") pod \"iptables-alerter-sdhv4\" (UID: \"45621191-5080-4ee9-a08e-f7b2ee49fe41\") " pod="openshift-network-operator/iptables-alerter-sdhv4"
Nov 04 03:17:38.758087 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.758066    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hp4bb\" (UniqueName: \"kubernetes.io/projected/7713890e-763d-4a51-bffb-6cf8dd12fd18-kube-api-access-hp4bb\") pod \"node-ca-n4b5s\" (UID: \"7713890e-763d-4a51-bffb-6cf8dd12fd18\") " pod="openshift-image-registry/node-ca-n4b5s"
Nov 04 03:17:38.758202 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.758183    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbrl5\" (UniqueName: \"kubernetes.io/projected/e87dded5-1e1b-45cb-ab2e-ae12550dfca7-kube-api-access-sbrl5\") pod \"cluster-monitoring-operator-884576c57-gmt9g\" (UID: \"e87dded5-1e1b-45cb-ab2e-ae12550dfca7\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmt9g"
Nov 04 03:17:38.758236 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.758223    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jltvv\" (UniqueName: \"kubernetes.io/projected/04b955ef-528b-4573-b18f-0957658f79e4-kube-api-access-jltvv\") pod \"network-check-target-hjssc\" (UID: \"04b955ef-528b-4573-b18f-0957658f79e4\") " pod="openshift-network-diagnostics/network-check-target-hjssc"
Nov 04 03:17:38.758481 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.758466    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f1f61b42-3b07-4892-90a7-2efa024e435b-bound-sa-token\") pod \"image-registry-79d6dd87ff-n2jnc\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") " pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:17:38.759709 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.759687    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fhxt\" (UniqueName: \"kubernetes.io/projected/536a0057-a438-4ad9-a4cd-b26cb7f3c645-kube-api-access-7fhxt\") pod \"insights-runtime-extractor-46btb\" (UID: \"536a0057-a438-4ad9-a4cd-b26cb7f3c645\") " pod="openshift-insights/insights-runtime-extractor-46btb"
Nov 04 03:17:38.759749 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.759729    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-66hms\" (UniqueName: \"kubernetes.io/projected/e0c72601-2624-4632-9f15-c19fcc489aae-kube-api-access-66hms\") pod \"kas-connection-checker-685895c7c9-mdchx\" (UID: \"e0c72601-2624-4632-9f15-c19fcc489aae\") " pod="kube-system/kas-connection-checker-685895c7c9-mdchx"
Nov 04 03:17:38.762089 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.762073    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5krfn\" (UniqueName: \"kubernetes.io/projected/fc3d3c0e-2877-4264-93a7-be3756d2dc5c-kube-api-access-5krfn\") pod \"network-metrics-daemon-lngrr\" (UID: \"fc3d3c0e-2877-4264-93a7-be3756d2dc5c\") " pod="openshift-multus/network-metrics-daemon-lngrr"
Nov 04 03:17:38.764448 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.764125    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qm4f8\" (UniqueName: \"kubernetes.io/projected/a683bd45-33c3-43fb-9530-292d7fea6970-kube-api-access-qm4f8\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:17:38.764448 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.764137    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ldp7\" (UniqueName: \"kubernetes.io/projected/a51f26a7-0ec8-4c56-881a-98c6d625478b-kube-api-access-2ldp7\") pod \"kas-connection-checker-685895c7c9-xpmwt\" (UID: \"a51f26a7-0ec8-4c56-881a-98c6d625478b\") " pod="kube-system/kas-connection-checker-685895c7c9-xpmwt"
Nov 04 03:17:38.764448 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.764369    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fc612506-b699-44bc-a81b-06fee4e819fe-bound-sa-token\") pod \"image-registry-7cb9f7c89d-wnpll\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") " pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:17:38.764665 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.764451    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9trx7\" (UniqueName: \"kubernetes.io/projected/21a42913-5c4e-4057-aae5-2804038f16e4-kube-api-access-9trx7\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.764665 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.764457    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5cdl\" (UniqueName: \"kubernetes.io/projected/d604b334-07f4-40f2-bfa9-4b26c301c84c-kube-api-access-n5cdl\") pod \"aws-ebs-csi-driver-node-jvtr8\" (UID: \"d604b334-07f4-40f2-bfa9-4b26c301c84c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8"
Nov 04 03:17:38.764665 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.764645    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4sqp\" (UniqueName: \"kubernetes.io/projected/06698c23-887e-4ee5-9728-04098200c13d-kube-api-access-v4sqp\") pod \"dns-default-s4852\" (UID: \"06698c23-887e-4ee5-9728-04098200c13d\") " pod="openshift-dns/dns-default-s4852"
Nov 04 03:17:38.764665 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.764653    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7cm5\" (UniqueName: \"kubernetes.io/projected/aab0bd9b-cb90-4d9b-9f46-52c6c87a34cf-kube-api-access-b7cm5\") pod \"node-resolver-6pwhj\" (UID: \"aab0bd9b-cb90-4d9b-9f46-52c6c87a34cf\") " pod="openshift-dns/node-resolver-6pwhj"
Nov 04 03:17:38.764901 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.764821    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghplp\" (UniqueName: \"kubernetes.io/projected/3b178ce9-e140-4420-81f1-627de00b90c6-kube-api-access-ghplp\") pod \"insights-operator-7c4986d449-hsb49\" (UID: \"3b178ce9-e140-4420-81f1-627de00b90c6\") " pod="openshift-insights/insights-operator-7c4986d449-hsb49"
Nov 04 03:17:38.764951 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.764929    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcm4d\" (UniqueName: \"kubernetes.io/projected/068619f8-7fbb-4b22-ac7d-a1f144e084c0-kube-api-access-mcm4d\") pod \"console-operator-7cd56b8dbc-p9989\" (UID: \"068619f8-7fbb-4b22-ac7d-a1f144e084c0\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-p9989"
Nov 04 03:17:38.765624 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.765605    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfc98\" (UniqueName: \"kubernetes.io/projected/fc612506-b699-44bc-a81b-06fee4e819fe-kube-api-access-bfc98\") pod \"image-registry-7cb9f7c89d-wnpll\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") " pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:17:38.769220 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.769200    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-597ml\" (UniqueName: \"kubernetes.io/projected/10cd06d4-ef1a-4ecf-a72c-e6d0084f3032-kube-api-access-597ml\") pod \"kas-connection-checker-685895c7c9-n6r7s\" (UID: \"10cd06d4-ef1a-4ecf-a72c-e6d0084f3032\") " pod="kube-system/kas-connection-checker-685895c7c9-n6r7s"
Nov 04 03:17:38.769339 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.769321    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5h469\" (UniqueName: \"kubernetes.io/projected/55c4af4c-188b-495b-9052-5700a1f60602-kube-api-access-5h469\") pod \"ingress-canary-qbskg\" (UID: \"55c4af4c-188b-495b-9052-5700a1f60602\") " pod="openshift-ingress-canary/ingress-canary-qbskg"
Nov 04 03:17:38.802093 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.802061    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-hostroot\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:17:38.802181 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.802098    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-host-kubelet\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.802181 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.802117    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-host\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:17:38.802181 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.802137    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-multus-conf-dir\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:17:38.802181 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.802165    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-hostroot\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:17:38.802340 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.802174    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-host-kubelet\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.802340 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.802167    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-etc-sysctl-d\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:17:38.802340 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.802210    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-host\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:17:38.802340 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.802228    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-multus-conf-dir\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:17:38.802340 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.802215    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-etc-sysconfig\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:17:38.802340 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.802252    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-etc-sysconfig\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:17:38.802340 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.802255    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ff9f5afb-957c-4383-8fa9-698f7ada29b0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-67bjk\" (UID: \"ff9f5afb-957c-4383-8fa9-698f7ada29b0\") " pod="openshift-multus/multus-additional-cni-plugins-67bjk"
Nov 04 03:17:38.802340 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.802278    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-multus-cni-dir\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:17:38.802340 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.802294    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-etc-sysctl-d\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:17:38.802340 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.802297    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ff9f5afb-957c-4383-8fa9-698f7ada29b0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-67bjk\" (UID: \"ff9f5afb-957c-4383-8fa9-698f7ada29b0\") " pod="openshift-multus/multus-additional-cni-plugins-67bjk"
Nov 04 03:17:38.802340 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.802342    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-multus-cni-dir\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:17:38.802678 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.802391    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-run-systemd\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.802709 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.802296    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-run-systemd\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.802740 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.802722    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d10e9268-df07-428b-ae25-528d9fe075d2-dbus\") pod \"global-pull-secret-syncer-tkkhk\" (UID: \"d10e9268-df07-428b-ae25-528d9fe075d2\") " pod="kube-system/global-pull-secret-syncer-tkkhk"
Nov 04 03:17:38.802770 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.802744    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-etc-sysctl-conf\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:17:38.802770 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.802761    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-host-run-k8s-cni-cncf-io\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:17:38.802823 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.802776    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-run-ovn\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.802823 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.802791    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-host-run-multus-certs\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:17:38.802823 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.802807    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d604b334-07f4-40f2-bfa9-4b26c301c84c-registration-dir\") pod \"aws-ebs-csi-driver-node-jvtr8\" (UID: \"d604b334-07f4-40f2-bfa9-4b26c301c84c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8"
Nov 04 03:17:38.802930 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.802828    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-etc-kubernetes\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:17:38.802930 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.802834    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-run-ovn\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.802930 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.802846    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d10e9268-df07-428b-ae25-528d9fe075d2-dbus\") pod \"global-pull-secret-syncer-tkkhk\" (UID: \"d10e9268-df07-428b-ae25-528d9fe075d2\") " pod="kube-system/global-pull-secret-syncer-tkkhk"
Nov 04 03:17:38.802930 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.802851    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-host-run-k8s-cni-cncf-io\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:17:38.803184 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.802896    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-etc-sysctl-conf\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:17:38.803264 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.803197    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-systemd-units\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.803264 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.802845    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-systemd-units\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.803264 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.803235    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-host-run-multus-certs\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:17:38.803264 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.803244    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d604b334-07f4-40f2-bfa9-4b26c301c84c-registration-dir\") pod \"aws-ebs-csi-driver-node-jvtr8\" (UID: \"d604b334-07f4-40f2-bfa9-4b26c301c84c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8"
Nov 04 03:17:38.803487 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.803243    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-etc-kubernetes\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:17:38.803487 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.803272    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-var-lib-openvswitch\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.803487 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.803308    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-sys\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:17:38.803487 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.803335    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-var-lib-openvswitch\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.803487 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.803369    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-host-run-netns\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:17:38.803487 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.803378    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-sys\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:17:38.803487 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.803436    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-run\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:17:38.803487 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.803470    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-host-var-lib-cni-bin\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:17:38.803883 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.803517    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-host-run-netns\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:17:38.803883 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.803532    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-host-var-lib-cni-bin\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:17:38.803883 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.803591    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-run\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:17:38.803883 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.803635    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d10e9268-df07-428b-ae25-528d9fe075d2-kubelet-config\") pod \"global-pull-secret-syncer-tkkhk\" (UID: \"d10e9268-df07-428b-ae25-528d9fe075d2\") " pod="kube-system/global-pull-secret-syncer-tkkhk"
Nov 04 03:17:38.803883 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.803669    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-host-var-lib-kubelet\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:17:38.803883 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.803697    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-host-run-ovn-kubernetes\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.803883 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.803729    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d604b334-07f4-40f2-bfa9-4b26c301c84c-device-dir\") pod \"aws-ebs-csi-driver-node-jvtr8\" (UID: \"d604b334-07f4-40f2-bfa9-4b26c301c84c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8"
Nov 04 03:17:38.803883 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.803761    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ff9f5afb-957c-4383-8fa9-698f7ada29b0-system-cni-dir\") pod \"multus-additional-cni-plugins-67bjk\" (UID: \"ff9f5afb-957c-4383-8fa9-698f7ada29b0\") " pod="openshift-multus/multus-additional-cni-plugins-67bjk"
Nov 04 03:17:38.803883 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.803791    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ff9f5afb-957c-4383-8fa9-698f7ada29b0-cnibin\") pod \"multus-additional-cni-plugins-67bjk\" (UID: \"ff9f5afb-957c-4383-8fa9-698f7ada29b0\") " pod="openshift-multus/multus-additional-cni-plugins-67bjk"
Nov 04 03:17:38.803883 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.803820    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d604b334-07f4-40f2-bfa9-4b26c301c84c-etc-selinux\") pod \"aws-ebs-csi-driver-node-jvtr8\" (UID: \"d604b334-07f4-40f2-bfa9-4b26c301c84c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8"
Nov 04 03:17:38.803883 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.803851    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/aab0bd9b-cb90-4d9b-9f46-52c6c87a34cf-hosts-file\") pod \"node-resolver-6pwhj\" (UID: \"aab0bd9b-cb90-4d9b-9f46-52c6c87a34cf\") " pod="openshift-dns/node-resolver-6pwhj"
Nov 04 03:17:38.803883 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.803882    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-system-cni-dir\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:17:38.804388 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.803916    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-host-slash\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.804388 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.803946    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-run-openvswitch\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.804388 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.803972    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-node-log\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.804388 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.803999    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-host-cni-bin\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.804388 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.804029    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.804388 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.804063    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-log-socket\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.804388 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.804094    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/536a0057-a438-4ad9-a4cd-b26cb7f3c645-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-46btb\" (UID: \"536a0057-a438-4ad9-a4cd-b26cb7f3c645\") " pod="openshift-insights/insights-runtime-extractor-46btb"
Nov 04 03:17:38.804388 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.804121    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-etc-openvswitch\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.804388 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.804149    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-var-lib-kubelet\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:17:38.804388 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.804177    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-multus-socket-dir-parent\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:17:38.804388 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.804210    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/45621191-5080-4ee9-a08e-f7b2ee49fe41-host-slash\") pod \"iptables-alerter-sdhv4\" (UID: \"45621191-5080-4ee9-a08e-f7b2ee49fe41\") " pod="openshift-network-operator/iptables-alerter-sdhv4"
Nov 04 03:17:38.804388 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.804233    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d604b334-07f4-40f2-bfa9-4b26c301c84c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jvtr8\" (UID: \"d604b334-07f4-40f2-bfa9-4b26c301c84c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8"
Nov 04 03:17:38.804388 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.804263    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d604b334-07f4-40f2-bfa9-4b26c301c84c-socket-dir\") pod \"aws-ebs-csi-driver-node-jvtr8\" (UID: \"d604b334-07f4-40f2-bfa9-4b26c301c84c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8"
Nov 04 03:17:38.804388 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.804293    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-host-var-lib-cni-multus\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:17:38.804388 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.804324    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-cnibin\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:17:38.804388 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.804356    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-etc-systemd\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:17:38.804388 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.804380    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-lib-modules\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:17:38.805109 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.804429    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-host-run-netns\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.805109 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.804469    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-etc-kubernetes\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:17:38.805109 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.804503    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-host-cni-netd\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.805109 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.804531    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/536a0057-a438-4ad9-a4cd-b26cb7f3c645-crio-socket\") pod \"insights-runtime-extractor-46btb\" (UID: \"536a0057-a438-4ad9-a4cd-b26cb7f3c645\") " pod="openshift-insights/insights-runtime-extractor-46btb"
Nov 04 03:17:38.805109 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.804561    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-etc-modprobe-d\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:17:38.805109 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.804602    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7713890e-763d-4a51-bffb-6cf8dd12fd18-host\") pod \"node-ca-n4b5s\" (UID: \"7713890e-763d-4a51-bffb-6cf8dd12fd18\") " pod="openshift-image-registry/node-ca-n4b5s"
Nov 04 03:17:38.805109 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.804634    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d604b334-07f4-40f2-bfa9-4b26c301c84c-sys-fs\") pod \"aws-ebs-csi-driver-node-jvtr8\" (UID: \"d604b334-07f4-40f2-bfa9-4b26c301c84c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8"
Nov 04 03:17:38.805109 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.804772    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d604b334-07f4-40f2-bfa9-4b26c301c84c-sys-fs\") pod \"aws-ebs-csi-driver-node-jvtr8\" (UID: \"d604b334-07f4-40f2-bfa9-4b26c301c84c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8"
Nov 04 03:17:38.805109 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.804843    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d10e9268-df07-428b-ae25-528d9fe075d2-kubelet-config\") pod \"global-pull-secret-syncer-tkkhk\" (UID: \"d10e9268-df07-428b-ae25-528d9fe075d2\") " pod="kube-system/global-pull-secret-syncer-tkkhk"
Nov 04 03:17:38.805109 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.804909    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-host-var-lib-kubelet\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:17:38.805109 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.804957    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-host-run-ovn-kubernetes\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.805109 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.805010    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d604b334-07f4-40f2-bfa9-4b26c301c84c-device-dir\") pod \"aws-ebs-csi-driver-node-jvtr8\" (UID: \"d604b334-07f4-40f2-bfa9-4b26c301c84c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8"
Nov 04 03:17:38.805109 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.805058    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ff9f5afb-957c-4383-8fa9-698f7ada29b0-system-cni-dir\") pod \"multus-additional-cni-plugins-67bjk\" (UID: \"ff9f5afb-957c-4383-8fa9-698f7ada29b0\") " pod="openshift-multus/multus-additional-cni-plugins-67bjk"
Nov 04 03:17:38.805109 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.805112    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ff9f5afb-957c-4383-8fa9-698f7ada29b0-cnibin\") pod \"multus-additional-cni-plugins-67bjk\" (UID: \"ff9f5afb-957c-4383-8fa9-698f7ada29b0\") " pod="openshift-multus/multus-additional-cni-plugins-67bjk"
Nov 04 03:17:38.805749 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.805170    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d604b334-07f4-40f2-bfa9-4b26c301c84c-etc-selinux\") pod \"aws-ebs-csi-driver-node-jvtr8\" (UID: \"d604b334-07f4-40f2-bfa9-4b26c301c84c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8"
Nov 04 03:17:38.805749 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.805266    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/aab0bd9b-cb90-4d9b-9f46-52c6c87a34cf-hosts-file\") pod \"node-resolver-6pwhj\" (UID: \"aab0bd9b-cb90-4d9b-9f46-52c6c87a34cf\") " pod="openshift-dns/node-resolver-6pwhj"
Nov 04 03:17:38.805749 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.805321    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-system-cni-dir\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:17:38.805749 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.805372    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-host-slash\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.805749 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.805432    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-run-openvswitch\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.805749 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.805477    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-node-log\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.805749 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.805522    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-host-cni-bin\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.805749 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.805569    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.805749 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.805619    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-log-socket\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.806132 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.806024    5609 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 04 03:17:38.807207 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.806643    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-etc-openvswitch\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.807207 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.806739    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-var-lib-kubelet\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:17:38.807207 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.806809    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-multus-socket-dir-parent\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:17:38.807207 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.806864    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/45621191-5080-4ee9-a08e-f7b2ee49fe41-host-slash\") pod \"iptables-alerter-sdhv4\" (UID: \"45621191-5080-4ee9-a08e-f7b2ee49fe41\") " pod="openshift-network-operator/iptables-alerter-sdhv4"
Nov 04 03:17:38.807207 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.806922    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d604b334-07f4-40f2-bfa9-4b26c301c84c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jvtr8\" (UID: \"d604b334-07f4-40f2-bfa9-4b26c301c84c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8"
Nov 04 03:17:38.807207 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.806990    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d604b334-07f4-40f2-bfa9-4b26c301c84c-socket-dir\") pod \"aws-ebs-csi-driver-node-jvtr8\" (UID: \"d604b334-07f4-40f2-bfa9-4b26c301c84c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jvtr8"
Nov 04 03:17:38.807207 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.807048    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-host-run-netns\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.807207 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.807082    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-host-var-lib-cni-multus\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:17:38.807207 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.807138    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-etc-modprobe-d\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:17:38.807207 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.807147    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-lib-modules\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:17:38.807207 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.807172    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7713890e-763d-4a51-bffb-6cf8dd12fd18-host\") pod \"node-ca-n4b5s\" (UID: \"7713890e-763d-4a51-bffb-6cf8dd12fd18\") " pod="openshift-image-registry/node-ca-n4b5s"
Nov 04 03:17:38.807207 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.807167    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6f627240-24ef-4aee-9810-51f422083ca3-cnibin\") pod \"multus-9xgrv\" (UID: \"6f627240-24ef-4aee-9810-51f422083ca3\") " pod="openshift-multus/multus-9xgrv"
Nov 04 03:17:38.807207 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.807181    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-etc-systemd\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:17:38.807735 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.807255    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/21a42913-5c4e-4057-aae5-2804038f16e4-host-cni-netd\") pod \"ovnkube-node-vmw5z\" (UID: \"21a42913-5c4e-4057-aae5-2804038f16e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:38.807735 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.807292    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a683bd45-33c3-43fb-9530-292d7fea6970-etc-kubernetes\") pod \"tuned-qtt77\" (UID: \"a683bd45-33c3-43fb-9530-292d7fea6970\") " pod="openshift-cluster-node-tuning-operator/tuned-qtt77"
Nov 04 03:17:38.807735 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.807161    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/536a0057-a438-4ad9-a4cd-b26cb7f3c645-crio-socket\") pod \"insights-runtime-extractor-46btb\" (UID: \"536a0057-a438-4ad9-a4cd-b26cb7f3c645\") " pod="openshift-insights/insights-runtime-extractor-46btb"
Nov 04 03:17:38.808572 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.808544    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/536a0057-a438-4ad9-a4cd-b26cb7f3c645-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-46btb\" (UID: \"536a0057-a438-4ad9-a4cd-b26cb7f3c645\") " pod="openshift-insights/insights-runtime-extractor-46btb"
Nov 04 03:17:38.889046 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.889005    5609 scope.go:122] "RemoveContainer" containerID="92d2557ce0ed61ad88da7d5c6d76d645bf407cd2ce7733bd5f621f087bce4578"
Nov 04 03:17:38.889507 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.889477    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7cfb798487-kbfzk"
Nov 04 03:17:38.900692 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.900671    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-46btb"
Nov 04 03:17:38.910621 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:38.910596    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-s4852"
Nov 04 03:17:39.276909 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:39.276873    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-46btb"]
Nov 04 03:17:39.283831 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:39.283797    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod536a0057_a438_4ad9_a4cd_b26cb7f3c645.slice/crio-c4fba15823aeb34ae6749a6a826d8344ad8f64f0b56909ec954e0b6803d29374 WatchSource:0}: Error finding container c4fba15823aeb34ae6749a6a826d8344ad8f64f0b56909ec954e0b6803d29374: Status 404 returned error can't find the container with id c4fba15823aeb34ae6749a6a826d8344ad8f64f0b56909ec954e0b6803d29374
Nov 04 03:17:39.344686 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:39.344663    5609 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 03:17:39.605779 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:39.605705    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-p9989_068619f8-7fbb-4b22-ac7d-a1f144e084c0/console-operator/2.log"
Nov 04 03:17:39.606178 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:39.605787    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-p9989" event={"ID":"068619f8-7fbb-4b22-ac7d-a1f144e084c0","Type":"ContainerStarted","Data":"545245e6584b4dfbe135b453e6e6adf74cca1d0af21e99d011f60f6820ac11c0"}
Nov 04 03:17:39.607387 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:39.607361    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-46btb" event={"ID":"536a0057-a438-4ad9-a4cd-b26cb7f3c645","Type":"ContainerStarted","Data":"1968f4e21eb297b79428d5cf56c5f49fb0a015c4c201692c37905a8a1573e562"}
Nov 04 03:17:39.607492 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:39.607432    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-46btb" event={"ID":"536a0057-a438-4ad9-a4cd-b26cb7f3c645","Type":"ContainerStarted","Data":"c4fba15823aeb34ae6749a6a826d8344ad8f64f0b56909ec954e0b6803d29374"}
Nov 04 03:17:39.891327 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:39.891257    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7cfb798487-kbfzk"
Nov 04 03:17:40.435569 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:40.435540    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:40.612861 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:40.612779    5609 generic.go:356] "Generic (PLEG): container finished" podID="3b178ce9-e140-4420-81f1-627de00b90c6" containerID="79b17768c249ad4244c1d5a0fb2f9a7933644b30ade14ed06d2c721b7c397fe9" exitCode=0
Nov 04 03:17:40.613302 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:40.612859    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-hsb49" event={"ID":"3b178ce9-e140-4420-81f1-627de00b90c6","Type":"ContainerDied","Data":"79b17768c249ad4244c1d5a0fb2f9a7933644b30ade14ed06d2c721b7c397fe9"}
Nov 04 03:17:40.613302 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:40.613255    5609 scope.go:122] "RemoveContainer" containerID="79b17768c249ad4244c1d5a0fb2f9a7933644b30ade14ed06d2c721b7c397fe9"
Nov 04 03:17:40.614699 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:40.614670    5609 generic.go:356] "Generic (PLEG): container finished" podID="6dedccdc-aea2-4664-8480-ca3c9359437d" containerID="a76b21e698659465e2efb560a01e2d433a6be7674c32fed65285c373cf0f5d0a" exitCode=0
Nov 04 03:17:40.614809 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:40.614703    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dln2k" event={"ID":"6dedccdc-aea2-4664-8480-ca3c9359437d","Type":"ContainerDied","Data":"a76b21e698659465e2efb560a01e2d433a6be7674c32fed65285c373cf0f5d0a"}
Nov 04 03:17:40.615084 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:40.615051    5609 scope.go:122] "RemoveContainer" containerID="a76b21e698659465e2efb560a01e2d433a6be7674c32fed65285c373cf0f5d0a"
Nov 04 03:17:40.616833 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:40.616794    5609 generic.go:356] "Generic (PLEG): container finished" podID="0beb091e-5c2f-4136-93a2-771f9b861411" containerID="a6c981ad2470dd6f32e35ca04b810224a433e5915ee5d4d759e922e026a06479" exitCode=0
Nov 04 03:17:40.616927 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:40.616865    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-js5xd" event={"ID":"0beb091e-5c2f-4136-93a2-771f9b861411","Type":"ContainerDied","Data":"a6c981ad2470dd6f32e35ca04b810224a433e5915ee5d4d759e922e026a06479"}
Nov 04 03:17:40.617445 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:40.617425    5609 scope.go:122] "RemoveContainer" containerID="a6c981ad2470dd6f32e35ca04b810224a433e5915ee5d4d759e922e026a06479"
Nov 04 03:17:40.676596 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:40.676570    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:17:40.948851 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:40.948819    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7cfb798487-kbfzk"
Nov 04 03:17:40.950073 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:40.950052    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7cfb798487-kbfzk"
Nov 04 03:17:41.621876 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:41.621838    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-hsb49" event={"ID":"3b178ce9-e140-4420-81f1-627de00b90c6","Type":"ContainerStarted","Data":"4245edfa596df57bcf5738b96bd668fdf6a0443dac5154534590adbc1336d4b1"}
Nov 04 03:17:41.623898 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:41.623864    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-46btb" event={"ID":"536a0057-a438-4ad9-a4cd-b26cb7f3c645","Type":"ContainerStarted","Data":"8e68ed128c5fd245e2fe962ca0479f775a94f66ac542f14c0cd8b75ab8d648d4"}
Nov 04 03:17:41.625666 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:41.625638    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dln2k" event={"ID":"6dedccdc-aea2-4664-8480-ca3c9359437d","Type":"ContainerStarted","Data":"a7c07468b5b576a6738dec5b0e4d6445a08a64b68ab3eecec93d93a1e7ed7b05"}
Nov 04 03:17:41.627443 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:41.627395    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-js5xd" event={"ID":"0beb091e-5c2f-4136-93a2-771f9b861411","Type":"ContainerStarted","Data":"10a2a64176489719cf184942720005bde05b5a9cc340c0333f0a6a61d2e8e0b4"}
Nov 04 03:17:42.303087 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:42.303050    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-p9989"
Nov 04 03:17:42.303269 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:42.303203    5609 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 04 03:17:42.308377 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:42.308346    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-p9989"
Nov 04 03:17:43.636492 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:43.636452    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-46btb" event={"ID":"536a0057-a438-4ad9-a4cd-b26cb7f3c645","Type":"ContainerStarted","Data":"a48efa59614cf0257e6059af8d3c47570be13b4c6dae88f844771c916f7ee577"}
Nov 04 03:17:45.525585 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:45.525552    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:17:45.526022 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:45.525869    5609 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 04 03:17:45.732633 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:45.732602    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-hjssc"
Nov 04 03:17:47.981986 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:47.981886    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-rmxkn"]
Nov 04 03:17:47.986106 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:47.986076    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rmxkn"
Nov 04 03:17:48.073711 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:48.073678    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0866082f-d2ee-42e8-a599-f567af165fbe-node-exporter-textfile\") pod \"node-exporter-rmxkn\" (UID: \"0866082f-d2ee-42e8-a599-f567af165fbe\") " pod="openshift-monitoring/node-exporter-rmxkn"
Nov 04 03:17:48.073711 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:48.073714    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0866082f-d2ee-42e8-a599-f567af165fbe-node-exporter-accelerators-collector-config\") pod \"node-exporter-rmxkn\" (UID: \"0866082f-d2ee-42e8-a599-f567af165fbe\") " pod="openshift-monitoring/node-exporter-rmxkn"
Nov 04 03:17:48.073935 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:48.073743    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55ztx\" (UniqueName: \"kubernetes.io/projected/0866082f-d2ee-42e8-a599-f567af165fbe-kube-api-access-55ztx\") pod \"node-exporter-rmxkn\" (UID: \"0866082f-d2ee-42e8-a599-f567af165fbe\") " pod="openshift-monitoring/node-exporter-rmxkn"
Nov 04 03:17:48.073935 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:48.073769    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0866082f-d2ee-42e8-a599-f567af165fbe-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rmxkn\" (UID: \"0866082f-d2ee-42e8-a599-f567af165fbe\") " pod="openshift-monitoring/node-exporter-rmxkn"
Nov 04 03:17:48.073935 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:48.073905    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0866082f-d2ee-42e8-a599-f567af165fbe-root\") pod \"node-exporter-rmxkn\" (UID: \"0866082f-d2ee-42e8-a599-f567af165fbe\") " pod="openshift-monitoring/node-exporter-rmxkn"
Nov 04 03:17:48.074070 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:48.073942    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0866082f-d2ee-42e8-a599-f567af165fbe-sys\") pod \"node-exporter-rmxkn\" (UID: \"0866082f-d2ee-42e8-a599-f567af165fbe\") " pod="openshift-monitoring/node-exporter-rmxkn"
Nov 04 03:17:48.074070 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:48.073963    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0866082f-d2ee-42e8-a599-f567af165fbe-node-exporter-tls\") pod \"node-exporter-rmxkn\" (UID: \"0866082f-d2ee-42e8-a599-f567af165fbe\") " pod="openshift-monitoring/node-exporter-rmxkn"
Nov 04 03:17:48.074070 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:48.073977    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0866082f-d2ee-42e8-a599-f567af165fbe-node-exporter-wtmp\") pod \"node-exporter-rmxkn\" (UID: \"0866082f-d2ee-42e8-a599-f567af165fbe\") " pod="openshift-monitoring/node-exporter-rmxkn"
Nov 04 03:17:48.074215 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:48.074102    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0866082f-d2ee-42e8-a599-f567af165fbe-metrics-client-ca\") pod \"node-exporter-rmxkn\" (UID: \"0866082f-d2ee-42e8-a599-f567af165fbe\") " pod="openshift-monitoring/node-exporter-rmxkn"
Nov 04 03:17:48.110975 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:48.110939    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Nov 04 03:17:48.111129 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:48.111096    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Nov 04 03:17:48.111190 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:48.111131    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-ctwv8\""
Nov 04 03:17:48.111190 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:48.111140    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Nov 04 03:17:48.158690 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:48.158645    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Nov 04 03:17:48.174906 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:48.174869    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0866082f-d2ee-42e8-a599-f567af165fbe-node-exporter-textfile\") pod \"node-exporter-rmxkn\" (UID: \"0866082f-d2ee-42e8-a599-f567af165fbe\") " pod="openshift-monitoring/node-exporter-rmxkn"
Nov 04 03:17:48.174906 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:48.174906    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0866082f-d2ee-42e8-a599-f567af165fbe-node-exporter-accelerators-collector-config\") pod \"node-exporter-rmxkn\" (UID: \"0866082f-d2ee-42e8-a599-f567af165fbe\") " pod="openshift-monitoring/node-exporter-rmxkn"
Nov 04 03:17:48.175240 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:48.174932    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55ztx\" (UniqueName: \"kubernetes.io/projected/0866082f-d2ee-42e8-a599-f567af165fbe-kube-api-access-55ztx\") pod \"node-exporter-rmxkn\" (UID: \"0866082f-d2ee-42e8-a599-f567af165fbe\") " pod="openshift-monitoring/node-exporter-rmxkn"
Nov 04 03:17:48.175240 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:48.175062    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0866082f-d2ee-42e8-a599-f567af165fbe-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rmxkn\" (UID: \"0866082f-d2ee-42e8-a599-f567af165fbe\") " pod="openshift-monitoring/node-exporter-rmxkn"
Nov 04 03:17:48.175240 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:48.175151    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0866082f-d2ee-42e8-a599-f567af165fbe-root\") pod \"node-exporter-rmxkn\" (UID: \"0866082f-d2ee-42e8-a599-f567af165fbe\") " pod="openshift-monitoring/node-exporter-rmxkn"
Nov 04 03:17:48.175240 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:48.175188    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0866082f-d2ee-42e8-a599-f567af165fbe-sys\") pod \"node-exporter-rmxkn\" (UID: \"0866082f-d2ee-42e8-a599-f567af165fbe\") " pod="openshift-monitoring/node-exporter-rmxkn"
Nov 04 03:17:48.175240 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:48.175217    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0866082f-d2ee-42e8-a599-f567af165fbe-node-exporter-tls\") pod \"node-exporter-rmxkn\" (UID: \"0866082f-d2ee-42e8-a599-f567af165fbe\") " pod="openshift-monitoring/node-exporter-rmxkn"
Nov 04 03:17:48.175240 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:48.175242    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0866082f-d2ee-42e8-a599-f567af165fbe-node-exporter-wtmp\") pod \"node-exporter-rmxkn\" (UID: \"0866082f-d2ee-42e8-a599-f567af165fbe\") " pod="openshift-monitoring/node-exporter-rmxkn"
Nov 04 03:17:48.175577 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:48.175267    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0866082f-d2ee-42e8-a599-f567af165fbe-node-exporter-textfile\") pod \"node-exporter-rmxkn\" (UID: \"0866082f-d2ee-42e8-a599-f567af165fbe\") " pod="openshift-monitoring/node-exporter-rmxkn"
Nov 04 03:17:48.175577 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:48.175269    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0866082f-d2ee-42e8-a599-f567af165fbe-sys\") pod \"node-exporter-rmxkn\" (UID: \"0866082f-d2ee-42e8-a599-f567af165fbe\") " pod="openshift-monitoring/node-exporter-rmxkn"
Nov 04 03:17:48.175577 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:48.175344    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0866082f-d2ee-42e8-a599-f567af165fbe-metrics-client-ca\") pod \"node-exporter-rmxkn\" (UID: \"0866082f-d2ee-42e8-a599-f567af165fbe\") " pod="openshift-monitoring/node-exporter-rmxkn"
Nov 04 03:17:48.175577 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:48.175416    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0866082f-d2ee-42e8-a599-f567af165fbe-node-exporter-wtmp\") pod \"node-exporter-rmxkn\" (UID: \"0866082f-d2ee-42e8-a599-f567af165fbe\") " pod="openshift-monitoring/node-exporter-rmxkn"
Nov 04 03:17:48.175774 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:48.175597    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0866082f-d2ee-42e8-a599-f567af165fbe-root\") pod \"node-exporter-rmxkn\" (UID: \"0866082f-d2ee-42e8-a599-f567af165fbe\") " pod="openshift-monitoring/node-exporter-rmxkn"
Nov 04 03:17:48.175774 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:48.175727    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0866082f-d2ee-42e8-a599-f567af165fbe-node-exporter-accelerators-collector-config\") pod \"node-exporter-rmxkn\" (UID: \"0866082f-d2ee-42e8-a599-f567af165fbe\") " pod="openshift-monitoring/node-exporter-rmxkn"
Nov 04 03:17:48.175874 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:48.175810    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0866082f-d2ee-42e8-a599-f567af165fbe-metrics-client-ca\") pod \"node-exporter-rmxkn\" (UID: \"0866082f-d2ee-42e8-a599-f567af165fbe\") " pod="openshift-monitoring/node-exporter-rmxkn"
Nov 04 03:17:48.177519 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:48.177497    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0866082f-d2ee-42e8-a599-f567af165fbe-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rmxkn\" (UID: \"0866082f-d2ee-42e8-a599-f567af165fbe\") " pod="openshift-monitoring/node-exporter-rmxkn"
Nov 04 03:17:48.177707 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:48.177687    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0866082f-d2ee-42e8-a599-f567af165fbe-node-exporter-tls\") pod \"node-exporter-rmxkn\" (UID: \"0866082f-d2ee-42e8-a599-f567af165fbe\") " pod="openshift-monitoring/node-exporter-rmxkn"
Nov 04 03:17:48.233480 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:48.233396    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-55ztx\" (UniqueName: \"kubernetes.io/projected/0866082f-d2ee-42e8-a599-f567af165fbe-kube-api-access-55ztx\") pod \"node-exporter-rmxkn\" (UID: \"0866082f-d2ee-42e8-a599-f567af165fbe\") " pod="openshift-monitoring/node-exporter-rmxkn"
Nov 04 03:17:48.299767 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:48.299727    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rmxkn"
Nov 04 03:17:48.311824 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:17:48.311797    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0866082f_d2ee_42e8_a599_f567af165fbe.slice/crio-94e28fd92b0efd4aa131d94cec934711779b51c9de8f90a99c2de4ac201fbdd5 WatchSource:0}: Error finding container 94e28fd92b0efd4aa131d94cec934711779b51c9de8f90a99c2de4ac201fbdd5: Status 404 returned error can't find the container with id 94e28fd92b0efd4aa131d94cec934711779b51c9de8f90a99c2de4ac201fbdd5
Nov 04 03:17:48.560636 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:17:48.560551    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:17:48.655620 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:48.655585    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rmxkn" event={"ID":"0866082f-d2ee-42e8-a599-f567af165fbe","Type":"ContainerStarted","Data":"94e28fd92b0efd4aa131d94cec934711779b51c9de8f90a99c2de4ac201fbdd5"}
Nov 04 03:17:50.665588 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:50.665550    5609 generic.go:356] "Generic (PLEG): container finished" podID="0866082f-d2ee-42e8-a599-f567af165fbe" containerID="603303fefdde50524825170422cbb275da0d50161a5715e6e3d817343a8ff4c8" exitCode=0
Nov 04 03:17:50.666022 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:50.665610    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rmxkn" event={"ID":"0866082f-d2ee-42e8-a599-f567af165fbe","Type":"ContainerDied","Data":"603303fefdde50524825170422cbb275da0d50161a5715e6e3d817343a8ff4c8"}
Nov 04 03:17:51.671649 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:51.671611    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rmxkn" event={"ID":"0866082f-d2ee-42e8-a599-f567af165fbe","Type":"ContainerStarted","Data":"3fff97013eddb7f4919ea72b929112e888d812d9816ced9f45a7e9f79088a382"}
Nov 04 03:17:51.672055 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:51.671676    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rmxkn" event={"ID":"0866082f-d2ee-42e8-a599-f567af165fbe","Type":"ContainerStarted","Data":"5f66f12fb18c1a6cfb20705f5c7ce2814b3104e814637900aea8d8b132e05c2b"}
Nov 04 03:17:51.787018 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:51.786954    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-rmxkn" podStartSLOduration=3.007328212 podStartE2EDuration="4.786936126s" totalImagesPullingTime="1.709651532s" totalInitContainerRuntime="69.956382ms" isStatefulPod=true podCreationTimestamp="2026-11-04 03:17:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:17:51.786741381 +0000 UTC m=+14.363394299" watchObservedRunningTime="2026-11-04 03:17:51.786936126 +0000 UTC m=+14.363589038"
Nov 04 03:17:54.619729 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:17:54.619695    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-79d6dd87ff-n2jnc"]
Nov 04 03:18:03.610878 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:03.610835    5609 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll" podUID="fc612506-b699-44bc-a81b-06fee4e819fe" containerName="registry" containerID="cri-o://5896ef33504444ee786b2836a14d4aaf1f015c80c7ed4a72e6918338be5342a0" gracePeriod=30
Nov 04 03:18:03.867086 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:03.867023    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:18:04.011424 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:04.011373    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fc612506-b699-44bc-a81b-06fee4e819fe-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fc612506-b699-44bc-a81b-06fee4e819fe-trusted-ca\") pod \"fc612506-b699-44bc-a81b-06fee4e819fe\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") "
Nov 04 03:18:04.011636 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:04.011447    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fc612506-b699-44bc-a81b-06fee4e819fe-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fc612506-b699-44bc-a81b-06fee4e819fe-registry-certificates\") pod \"fc612506-b699-44bc-a81b-06fee4e819fe\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") "
Nov 04 03:18:04.011636 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:04.011475    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc612506-b699-44bc-a81b-06fee4e819fe-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fc612506-b699-44bc-a81b-06fee4e819fe-ca-trust-extracted\") pod \"fc612506-b699-44bc-a81b-06fee4e819fe\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") "
Nov 04 03:18:04.011636 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:04.011530    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fc612506-b699-44bc-a81b-06fee4e819fe-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fc612506-b699-44bc-a81b-06fee4e819fe-installation-pull-secrets\") pod \"fc612506-b699-44bc-a81b-06fee4e819fe\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") "
Nov 04 03:18:04.011636 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:04.011586    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fc612506-b699-44bc-a81b-06fee4e819fe-kube-api-access-bfc98\" (UniqueName: \"kubernetes.io/projected/fc612506-b699-44bc-a81b-06fee4e819fe-kube-api-access-bfc98\") pod \"fc612506-b699-44bc-a81b-06fee4e819fe\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") "
Nov 04 03:18:04.011636 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:04.011636    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fc612506-b699-44bc-a81b-06fee4e819fe-registry-tls\" (UniqueName: \"kubernetes.io/projected/fc612506-b699-44bc-a81b-06fee4e819fe-registry-tls\") pod \"fc612506-b699-44bc-a81b-06fee4e819fe\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") "
Nov 04 03:18:04.011886 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:04.011678    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fc612506-b699-44bc-a81b-06fee4e819fe-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fc612506-b699-44bc-a81b-06fee4e819fe-bound-sa-token\") pod \"fc612506-b699-44bc-a81b-06fee4e819fe\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") "
Nov 04 03:18:04.011886 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:04.011715    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fc612506-b699-44bc-a81b-06fee4e819fe-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fc612506-b699-44bc-a81b-06fee4e819fe-image-registry-private-configuration\") pod \"fc612506-b699-44bc-a81b-06fee4e819fe\" (UID: \"fc612506-b699-44bc-a81b-06fee4e819fe\") "
Nov 04 03:18:04.011886 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:04.011851    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc612506-b699-44bc-a81b-06fee4e819fe-trusted-ca" pod "fc612506-b699-44bc-a81b-06fee4e819fe" (UID: "fc612506-b699-44bc-a81b-06fee4e819fe"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:18:04.012042 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:04.011898    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc612506-b699-44bc-a81b-06fee4e819fe-registry-certificates" pod "fc612506-b699-44bc-a81b-06fee4e819fe" (UID: "fc612506-b699-44bc-a81b-06fee4e819fe"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:18:04.012042 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:04.011986    5609 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fc612506-b699-44bc-a81b-06fee4e819fe-trusted-ca\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:18:04.012042 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:04.012009    5609 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fc612506-b699-44bc-a81b-06fee4e819fe-registry-certificates\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:18:04.014392 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:04.014321    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc612506-b699-44bc-a81b-06fee4e819fe-bound-sa-token" pod "fc612506-b699-44bc-a81b-06fee4e819fe" (UID: "fc612506-b699-44bc-a81b-06fee4e819fe"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:18:04.014552 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:04.014521    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc612506-b699-44bc-a81b-06fee4e819fe-kube-api-access-bfc98" pod "fc612506-b699-44bc-a81b-06fee4e819fe" (UID: "fc612506-b699-44bc-a81b-06fee4e819fe"). InnerVolumeSpecName "kube-api-access-bfc98". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:18:04.014601 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:04.014561    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc612506-b699-44bc-a81b-06fee4e819fe-registry-tls" pod "fc612506-b699-44bc-a81b-06fee4e819fe" (UID: "fc612506-b699-44bc-a81b-06fee4e819fe"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:18:04.015510 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:04.015484    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc612506-b699-44bc-a81b-06fee4e819fe-image-registry-private-configuration" pod "fc612506-b699-44bc-a81b-06fee4e819fe" (UID: "fc612506-b699-44bc-a81b-06fee4e819fe"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:18:04.015600 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:04.015566    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc612506-b699-44bc-a81b-06fee4e819fe-installation-pull-secrets" pod "fc612506-b699-44bc-a81b-06fee4e819fe" (UID: "fc612506-b699-44bc-a81b-06fee4e819fe"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:18:04.018676 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:04.018601    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc612506-b699-44bc-a81b-06fee4e819fe-ca-trust-extracted" pod "fc612506-b699-44bc-a81b-06fee4e819fe" (UID: "fc612506-b699-44bc-a81b-06fee4e819fe"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:18:04.112897 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:04.112861    5609 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fc612506-b699-44bc-a81b-06fee4e819fe-installation-pull-secrets\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:18:04.112897 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:04.112893    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfc98\" (UniqueName: \"kubernetes.io/projected/fc612506-b699-44bc-a81b-06fee4e819fe-kube-api-access-bfc98\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:18:04.112897 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:04.112903    5609 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fc612506-b699-44bc-a81b-06fee4e819fe-registry-tls\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:18:04.113130 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:04.112913    5609 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fc612506-b699-44bc-a81b-06fee4e819fe-bound-sa-token\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:18:04.113130 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:04.112923    5609 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fc612506-b699-44bc-a81b-06fee4e819fe-image-registry-private-configuration\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:18:04.113130 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:04.112932    5609 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fc612506-b699-44bc-a81b-06fee4e819fe-ca-trust-extracted\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:18:04.316223 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:04.316188    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-m2vkr"
Nov 04 03:18:04.592470 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:04.592375    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vmw5z"
Nov 04 03:18:04.723264 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:04.723231    5609 generic.go:356] "Generic (PLEG): container finished" podID="fc612506-b699-44bc-a81b-06fee4e819fe" containerID="5896ef33504444ee786b2836a14d4aaf1f015c80c7ed4a72e6918338be5342a0" exitCode=0
Nov 04 03:18:04.723697 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:04.723294    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll"
Nov 04 03:18:04.723697 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:04.723312    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll" event={"ID":"fc612506-b699-44bc-a81b-06fee4e819fe","Type":"ContainerDied","Data":"5896ef33504444ee786b2836a14d4aaf1f015c80c7ed4a72e6918338be5342a0"}
Nov 04 03:18:04.723697 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:04.723387    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7cb9f7c89d-wnpll" event={"ID":"fc612506-b699-44bc-a81b-06fee4e819fe","Type":"ContainerDied","Data":"b0d8f5dcca3cd28fec67127e455a70a4dd57c277871218844baaa61bc2e7b915"}
Nov 04 03:18:04.723697 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:04.723430    5609 scope.go:122] "RemoveContainer" containerID="5896ef33504444ee786b2836a14d4aaf1f015c80c7ed4a72e6918338be5342a0"
Nov 04 03:18:04.732541 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:04.732517    5609 scope.go:122] "RemoveContainer" containerID="5896ef33504444ee786b2836a14d4aaf1f015c80c7ed4a72e6918338be5342a0"
Nov 04 03:18:04.732860 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:18:04.732832    5609 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5896ef33504444ee786b2836a14d4aaf1f015c80c7ed4a72e6918338be5342a0\": container with ID starting with 5896ef33504444ee786b2836a14d4aaf1f015c80c7ed4a72e6918338be5342a0 not found: ID does not exist" containerID="5896ef33504444ee786b2836a14d4aaf1f015c80c7ed4a72e6918338be5342a0"
Nov 04 03:18:04.732924 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:04.732875    5609 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5896ef33504444ee786b2836a14d4aaf1f015c80c7ed4a72e6918338be5342a0"} err="failed to get container status \"5896ef33504444ee786b2836a14d4aaf1f015c80c7ed4a72e6918338be5342a0\": rpc error: code = NotFound desc = could not find container \"5896ef33504444ee786b2836a14d4aaf1f015c80c7ed4a72e6918338be5342a0\": container with ID starting with 5896ef33504444ee786b2836a14d4aaf1f015c80c7ed4a72e6918338be5342a0 not found: ID does not exist"
Nov 04 03:18:04.925730 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:04.925645    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7cb9f7c89d-wnpll"]
Nov 04 03:18:04.972900 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:04.972862    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7cb9f7c89d-wnpll"]
Nov 04 03:18:05.532929 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:05.532892    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fc612506-b699-44bc-a81b-06fee4e819fe" path="/var/lib/kubelet/pods/fc612506-b699-44bc-a81b-06fee4e819fe/volumes"
Nov 04 03:18:18.560234 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:18:18.560193    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:18:19.639556 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:19.639514    5609 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc" podUID="f1f61b42-3b07-4892-90a7-2efa024e435b" containerName="registry" containerID="cri-o://5140cf4d30301289198ac7d930beec469d79386b2c6405fb1a5c7ebcba51038d" gracePeriod=30
Nov 04 03:18:19.775717 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:19.775678    5609 generic.go:356] "Generic (PLEG): container finished" podID="f1f61b42-3b07-4892-90a7-2efa024e435b" containerID="5140cf4d30301289198ac7d930beec469d79386b2c6405fb1a5c7ebcba51038d" exitCode=0
Nov 04 03:18:19.775871 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:19.775748    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc" event={"ID":"f1f61b42-3b07-4892-90a7-2efa024e435b","Type":"ContainerDied","Data":"5140cf4d30301289198ac7d930beec469d79386b2c6405fb1a5c7ebcba51038d"}
Nov 04 03:18:19.899152 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:19.899089    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:18:20.040951 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:20.040918    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f1f61b42-3b07-4892-90a7-2efa024e435b-registry-tls\" (UniqueName: \"kubernetes.io/projected/f1f61b42-3b07-4892-90a7-2efa024e435b-registry-tls\") pod \"f1f61b42-3b07-4892-90a7-2efa024e435b\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") "
Nov 04 03:18:20.041154 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:20.040979    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f1f61b42-3b07-4892-90a7-2efa024e435b-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f1f61b42-3b07-4892-90a7-2efa024e435b-bound-sa-token\") pod \"f1f61b42-3b07-4892-90a7-2efa024e435b\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") "
Nov 04 03:18:20.041154 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:20.041012    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f1f61b42-3b07-4892-90a7-2efa024e435b-kube-api-access-kjtnj\" (UniqueName: \"kubernetes.io/projected/f1f61b42-3b07-4892-90a7-2efa024e435b-kube-api-access-kjtnj\") pod \"f1f61b42-3b07-4892-90a7-2efa024e435b\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") "
Nov 04 03:18:20.041154 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:20.041124    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f1f61b42-3b07-4892-90a7-2efa024e435b-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f1f61b42-3b07-4892-90a7-2efa024e435b-image-registry-private-configuration\") pod \"f1f61b42-3b07-4892-90a7-2efa024e435b\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") "
Nov 04 03:18:20.041319 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:20.041166    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f1f61b42-3b07-4892-90a7-2efa024e435b-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f1f61b42-3b07-4892-90a7-2efa024e435b-trusted-ca\") pod \"f1f61b42-3b07-4892-90a7-2efa024e435b\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") "
Nov 04 03:18:20.041319 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:20.041203    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f1f61b42-3b07-4892-90a7-2efa024e435b-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f1f61b42-3b07-4892-90a7-2efa024e435b-ca-trust-extracted\") pod \"f1f61b42-3b07-4892-90a7-2efa024e435b\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") "
Nov 04 03:18:20.041319 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:20.041223    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f1f61b42-3b07-4892-90a7-2efa024e435b-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f1f61b42-3b07-4892-90a7-2efa024e435b-registry-certificates\") pod \"f1f61b42-3b07-4892-90a7-2efa024e435b\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") "
Nov 04 03:18:20.041319 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:20.041260    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f1f61b42-3b07-4892-90a7-2efa024e435b-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f1f61b42-3b07-4892-90a7-2efa024e435b-installation-pull-secrets\") pod \"f1f61b42-3b07-4892-90a7-2efa024e435b\" (UID: \"f1f61b42-3b07-4892-90a7-2efa024e435b\") "
Nov 04 03:18:20.041847 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:20.041806    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1f61b42-3b07-4892-90a7-2efa024e435b-trusted-ca" pod "f1f61b42-3b07-4892-90a7-2efa024e435b" (UID: "f1f61b42-3b07-4892-90a7-2efa024e435b"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:18:20.041847 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:20.041802    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1f61b42-3b07-4892-90a7-2efa024e435b-registry-certificates" pod "f1f61b42-3b07-4892-90a7-2efa024e435b" (UID: "f1f61b42-3b07-4892-90a7-2efa024e435b"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:18:20.044061 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:20.044032    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1f61b42-3b07-4892-90a7-2efa024e435b-bound-sa-token" pod "f1f61b42-3b07-4892-90a7-2efa024e435b" (UID: "f1f61b42-3b07-4892-90a7-2efa024e435b"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:18:20.044168 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:20.044130    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1f61b42-3b07-4892-90a7-2efa024e435b-installation-pull-secrets" pod "f1f61b42-3b07-4892-90a7-2efa024e435b" (UID: "f1f61b42-3b07-4892-90a7-2efa024e435b"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:18:20.044233 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:20.044168    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1f61b42-3b07-4892-90a7-2efa024e435b-kube-api-access-kjtnj" pod "f1f61b42-3b07-4892-90a7-2efa024e435b" (UID: "f1f61b42-3b07-4892-90a7-2efa024e435b"). InnerVolumeSpecName "kube-api-access-kjtnj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:18:20.044233 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:20.044220    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1f61b42-3b07-4892-90a7-2efa024e435b-registry-tls" pod "f1f61b42-3b07-4892-90a7-2efa024e435b" (UID: "f1f61b42-3b07-4892-90a7-2efa024e435b"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:18:20.044327 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:20.044262    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1f61b42-3b07-4892-90a7-2efa024e435b-image-registry-private-configuration" pod "f1f61b42-3b07-4892-90a7-2efa024e435b" (UID: "f1f61b42-3b07-4892-90a7-2efa024e435b"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:18:20.048370 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:20.048340    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1f61b42-3b07-4892-90a7-2efa024e435b-ca-trust-extracted" pod "f1f61b42-3b07-4892-90a7-2efa024e435b" (UID: "f1f61b42-3b07-4892-90a7-2efa024e435b"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:18:20.142129 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:20.142095    5609 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f1f61b42-3b07-4892-90a7-2efa024e435b-trusted-ca\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:18:20.142129 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:20.142128    5609 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f1f61b42-3b07-4892-90a7-2efa024e435b-ca-trust-extracted\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:18:20.142330 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:20.142138    5609 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f1f61b42-3b07-4892-90a7-2efa024e435b-registry-certificates\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:18:20.142330 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:20.142149    5609 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f1f61b42-3b07-4892-90a7-2efa024e435b-installation-pull-secrets\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:18:20.142330 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:20.142158    5609 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f1f61b42-3b07-4892-90a7-2efa024e435b-registry-tls\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:18:20.142330 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:20.142181    5609 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f1f61b42-3b07-4892-90a7-2efa024e435b-bound-sa-token\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:18:20.142330 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:20.142191    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kjtnj\" (UniqueName: \"kubernetes.io/projected/f1f61b42-3b07-4892-90a7-2efa024e435b-kube-api-access-kjtnj\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:18:20.142330 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:20.142201    5609 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f1f61b42-3b07-4892-90a7-2efa024e435b-image-registry-private-configuration\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:18:20.781417 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:20.781371    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc"
Nov 04 03:18:20.781417 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:20.781391    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-79d6dd87ff-n2jnc" event={"ID":"f1f61b42-3b07-4892-90a7-2efa024e435b","Type":"ContainerDied","Data":"162b5150b1fa0c5491cabb76b570878f66758b46a19ba0e60cd6b5de9f2a4708"}
Nov 04 03:18:20.782070 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:20.781458    5609 scope.go:122] "RemoveContainer" containerID="5140cf4d30301289198ac7d930beec469d79386b2c6405fb1a5c7ebcba51038d"
Nov 04 03:18:20.984306 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:20.984268    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-79d6dd87ff-n2jnc"]
Nov 04 03:18:21.031966 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:21.031873    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-79d6dd87ff-n2jnc"]
Nov 04 03:18:21.532511 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:18:21.532476    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f1f61b42-3b07-4892-90a7-2efa024e435b" path="/var/lib/kubelet/pods/f1f61b42-3b07-4892-90a7-2efa024e435b/volumes"
Nov 04 03:18:48.559797 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:18:48.559758    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:19:18.562040 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:19:18.561947    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:19:48.560141 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:19:48.560093    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:20:18.559628 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:20:18.559588    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:20:48.559551 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:20:48.559459    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:21:18.559849 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:21:18.559815    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:21:48.558801 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:21:48.558765    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:22:18.560236 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:22:18.560150    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:22:37.543849 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:37.543813    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-p9989_068619f8-7fbb-4b22-ac7d-a1f144e084c0/console-operator/2.log"
Nov 04 03:22:37.547645 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:37.547625    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-p9989_068619f8-7fbb-4b22-ac7d-a1f144e084c0/console-operator/2.log"
Nov 04 03:22:37.551451 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:37.551434    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vmw5z_21a42913-5c4e-4057-aae5-2804038f16e4/ovn-acl-logging/0.log"
Nov 04 03:22:37.555063 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:37.555045    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vmw5z_21a42913-5c4e-4057-aae5-2804038f16e4/ovn-acl-logging/0.log"
Nov 04 03:22:48.559573 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:22:48.559528    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:22:51.200857 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:51.200819    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448d75f"]
Nov 04 03:22:51.201284 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:51.201136    5609 cpu_manager.go:414] "containerMap: removing container" podUID="f1f61b42-3b07-4892-90a7-2efa024e435b" containerName="registry"
Nov 04 03:22:51.201284 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:51.201149    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f1f61b42-3b07-4892-90a7-2efa024e435b" containerName="registry"
Nov 04 03:22:51.201284 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:51.201172    5609 cpu_manager.go:414] "containerMap: removing container" podUID="fc612506-b699-44bc-a81b-06fee4e819fe" containerName="registry"
Nov 04 03:22:51.201284 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:51.201177    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fc612506-b699-44bc-a81b-06fee4e819fe" containerName="registry"
Nov 04 03:22:51.201284 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:51.201223    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="f1f61b42-3b07-4892-90a7-2efa024e435b" containerName="registry"
Nov 04 03:22:51.201284 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:51.201234    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="fc612506-b699-44bc-a81b-06fee4e819fe" containerName="registry"
Nov 04 03:22:51.204347 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:51.204330    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448d75f"
Nov 04 03:22:51.280258 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:51.280221    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c65ca1f2-8c4e-420d-8726-4d01a567a302-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448d75f\" (UID: \"c65ca1f2-8c4e-420d-8726-4d01a567a302\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448d75f"
Nov 04 03:22:51.280441 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:51.280331    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c65ca1f2-8c4e-420d-8726-4d01a567a302-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448d75f\" (UID: \"c65ca1f2-8c4e-420d-8726-4d01a567a302\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448d75f"
Nov 04 03:22:51.280441 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:51.280396    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j25x8\" (UniqueName: \"kubernetes.io/projected/c65ca1f2-8c4e-420d-8726-4d01a567a302-kube-api-access-j25x8\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448d75f\" (UID: \"c65ca1f2-8c4e-420d-8726-4d01a567a302\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448d75f"
Nov 04 03:22:51.300947 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:51.300915    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Nov 04 03:22:51.300947 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:51.300912    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Nov 04 03:22:51.301141 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:51.300912    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c4959aeb960c8e20c88ac2de924a35b8221494-145dcd5a-dockercfg-z62zq\""
Nov 04 03:22:51.301897 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:51.301864    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448d75f"]
Nov 04 03:22:51.381279 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:51.381242    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c65ca1f2-8c4e-420d-8726-4d01a567a302-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448d75f\" (UID: \"c65ca1f2-8c4e-420d-8726-4d01a567a302\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448d75f"
Nov 04 03:22:51.381530 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:51.381329    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j25x8\" (UniqueName: \"kubernetes.io/projected/c65ca1f2-8c4e-420d-8726-4d01a567a302-kube-api-access-j25x8\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448d75f\" (UID: \"c65ca1f2-8c4e-420d-8726-4d01a567a302\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448d75f"
Nov 04 03:22:51.381530 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:51.381371    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c65ca1f2-8c4e-420d-8726-4d01a567a302-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448d75f\" (UID: \"c65ca1f2-8c4e-420d-8726-4d01a567a302\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448d75f"
Nov 04 03:22:51.381781 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:51.381753    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c65ca1f2-8c4e-420d-8726-4d01a567a302-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448d75f\" (UID: \"c65ca1f2-8c4e-420d-8726-4d01a567a302\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448d75f"
Nov 04 03:22:51.381781 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:51.381767    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c65ca1f2-8c4e-420d-8726-4d01a567a302-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448d75f\" (UID: \"c65ca1f2-8c4e-420d-8726-4d01a567a302\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448d75f"
Nov 04 03:22:51.382053 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:51.382034    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be712pstf"]
Nov 04 03:22:51.385042 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:51.385028    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be712pstf"
Nov 04 03:22:51.439045 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:51.439004    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j25x8\" (UniqueName: \"kubernetes.io/projected/c65ca1f2-8c4e-420d-8726-4d01a567a302-kube-api-access-j25x8\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448d75f\" (UID: \"c65ca1f2-8c4e-420d-8726-4d01a567a302\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448d75f"
Nov 04 03:22:51.442551 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:51.442521    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be712pstf"]
Nov 04 03:22:51.482087 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:51.482005    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c1a5a9af-8221-4655-ba83-7da2acf5244e-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be712pstf\" (UID: \"c1a5a9af-8221-4655-ba83-7da2acf5244e\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be712pstf"
Nov 04 03:22:51.482087 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:51.482059    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hntdh\" (UniqueName: \"kubernetes.io/projected/c1a5a9af-8221-4655-ba83-7da2acf5244e-kube-api-access-hntdh\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be712pstf\" (UID: \"c1a5a9af-8221-4655-ba83-7da2acf5244e\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be712pstf"
Nov 04 03:22:51.482262 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:51.482196    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c1a5a9af-8221-4655-ba83-7da2acf5244e-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be712pstf\" (UID: \"c1a5a9af-8221-4655-ba83-7da2acf5244e\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be712pstf"
Nov 04 03:22:51.510109 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:51.510082    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e644421527c169d876fdb59235c927986d6bd2-9d2815fc-dockercfg-6mvr8\""
Nov 04 03:22:51.520235 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:51.520208    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448d75f"
Nov 04 03:22:51.582956 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:51.582908    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c1a5a9af-8221-4655-ba83-7da2acf5244e-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be712pstf\" (UID: \"c1a5a9af-8221-4655-ba83-7da2acf5244e\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be712pstf"
Nov 04 03:22:51.583089 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:51.582988    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hntdh\" (UniqueName: \"kubernetes.io/projected/c1a5a9af-8221-4655-ba83-7da2acf5244e-kube-api-access-hntdh\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be712pstf\" (UID: \"c1a5a9af-8221-4655-ba83-7da2acf5244e\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be712pstf"
Nov 04 03:22:51.583089 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:51.583065    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c1a5a9af-8221-4655-ba83-7da2acf5244e-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be712pstf\" (UID: \"c1a5a9af-8221-4655-ba83-7da2acf5244e\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be712pstf"
Nov 04 03:22:51.583672 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:51.583638    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c1a5a9af-8221-4655-ba83-7da2acf5244e-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be712pstf\" (UID: \"c1a5a9af-8221-4655-ba83-7da2acf5244e\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be712pstf"
Nov 04 03:22:51.584918 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:51.583866    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c1a5a9af-8221-4655-ba83-7da2acf5244e-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be712pstf\" (UID: \"c1a5a9af-8221-4655-ba83-7da2acf5244e\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be712pstf"
Nov 04 03:22:51.641977 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:51.641949    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hntdh\" (UniqueName: \"kubernetes.io/projected/c1a5a9af-8221-4655-ba83-7da2acf5244e-kube-api-access-hntdh\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be712pstf\" (UID: \"c1a5a9af-8221-4655-ba83-7da2acf5244e\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be712pstf"
Nov 04 03:22:51.699108 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:51.699072    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be712pstf"
Nov 04 03:22:51.894530 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:51.894500    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448d75f"]
Nov 04 03:22:51.901137 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:22:51.901109    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc65ca1f2_8c4e_420d_8726_4d01a567a302.slice/crio-fbfdfbc540193717d15c8fc84398e58e697317932cf97a17daf175a12c1cd5ee WatchSource:0}: Error finding container fbfdfbc540193717d15c8fc84398e58e697317932cf97a17daf175a12c1cd5ee: Status 404 returned error can't find the container with id fbfdfbc540193717d15c8fc84398e58e697317932cf97a17daf175a12c1cd5ee
Nov 04 03:22:51.902903 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:51.902889    5609 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 03:22:52.075591 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:52.075515    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be712pstf"]
Nov 04 03:22:52.082526 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:22:52.082491    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1a5a9af_8221_4655_ba83_7da2acf5244e.slice/crio-eec74d167c20425871d2f402afde0251b2de2be7920fb5b10f8063a80b1cb1d9 WatchSource:0}: Error finding container eec74d167c20425871d2f402afde0251b2de2be7920fb5b10f8063a80b1cb1d9: Status 404 returned error can't find the container with id eec74d167c20425871d2f402afde0251b2de2be7920fb5b10f8063a80b1cb1d9
Nov 04 03:22:52.718728 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:52.718684    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448d75f" event={"ID":"c65ca1f2-8c4e-420d-8726-4d01a567a302","Type":"ContainerStarted","Data":"fbfdfbc540193717d15c8fc84398e58e697317932cf97a17daf175a12c1cd5ee"}
Nov 04 03:22:52.719699 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:52.719675    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be712pstf" event={"ID":"c1a5a9af-8221-4655-ba83-7da2acf5244e","Type":"ContainerStarted","Data":"eec74d167c20425871d2f402afde0251b2de2be7920fb5b10f8063a80b1cb1d9"}
Nov 04 03:22:53.659433 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:53.659385    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cf7j9g"]
Nov 04 03:22:53.662958 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:53.662930    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cf7j9g"
Nov 04 03:22:53.700919 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:53.700880    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4558182a-d695-4e09-8956-60c86a9c02f1-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cf7j9g\" (UID: \"4558182a-d695-4e09-8956-60c86a9c02f1\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cf7j9g"
Nov 04 03:22:53.701102 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:53.700940    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4558182a-d695-4e09-8956-60c86a9c02f1-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cf7j9g\" (UID: \"4558182a-d695-4e09-8956-60c86a9c02f1\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cf7j9g"
Nov 04 03:22:53.701102 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:53.701059    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nxw6\" (UniqueName: \"kubernetes.io/projected/4558182a-d695-4e09-8956-60c86a9c02f1-kube-api-access-9nxw6\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cf7j9g\" (UID: \"4558182a-d695-4e09-8956-60c86a9c02f1\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cf7j9g"
Nov 04 03:22:53.759841 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:53.759751    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c7537007944577f64df34960ace3c69136c416-139ec370-dockercfg-p7l5n\""
Nov 04 03:22:53.760846 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:53.760806    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cf7j9g"]
Nov 04 03:22:53.802302 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:53.802262    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4558182a-d695-4e09-8956-60c86a9c02f1-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cf7j9g\" (UID: \"4558182a-d695-4e09-8956-60c86a9c02f1\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cf7j9g"
Nov 04 03:22:53.802513 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:53.802323    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4558182a-d695-4e09-8956-60c86a9c02f1-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cf7j9g\" (UID: \"4558182a-d695-4e09-8956-60c86a9c02f1\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cf7j9g"
Nov 04 03:22:53.802513 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:53.802374    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nxw6\" (UniqueName: \"kubernetes.io/projected/4558182a-d695-4e09-8956-60c86a9c02f1-kube-api-access-9nxw6\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cf7j9g\" (UID: \"4558182a-d695-4e09-8956-60c86a9c02f1\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cf7j9g"
Nov 04 03:22:53.802786 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:53.802756    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4558182a-d695-4e09-8956-60c86a9c02f1-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cf7j9g\" (UID: \"4558182a-d695-4e09-8956-60c86a9c02f1\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cf7j9g"
Nov 04 03:22:53.802866 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:53.802789    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4558182a-d695-4e09-8956-60c86a9c02f1-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cf7j9g\" (UID: \"4558182a-d695-4e09-8956-60c86a9c02f1\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cf7j9g"
Nov 04 03:22:53.863199 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:53.863160    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nxw6\" (UniqueName: \"kubernetes.io/projected/4558182a-d695-4e09-8956-60c86a9c02f1-kube-api-access-9nxw6\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cf7j9g\" (UID: \"4558182a-d695-4e09-8956-60c86a9c02f1\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cf7j9g"
Nov 04 03:22:53.981495 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:53.981455    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cf7j9g"
Nov 04 03:22:54.354447 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:54.354353    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cf7j9g"]
Nov 04 03:22:54.361230 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:22:54.361193    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4558182a_d695_4e09_8956_60c86a9c02f1.slice/crio-f3d3f6ff2d1f656f425a04e1cf8f8d693491260d7f0641f40774143965ea267a WatchSource:0}: Error finding container f3d3f6ff2d1f656f425a04e1cf8f8d693491260d7f0641f40774143965ea267a: Status 404 returned error can't find the container with id f3d3f6ff2d1f656f425a04e1cf8f8d693491260d7f0641f40774143965ea267a
Nov 04 03:22:54.729080 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:54.729041    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cf7j9g" event={"ID":"4558182a-d695-4e09-8956-60c86a9c02f1","Type":"ContainerStarted","Data":"f3d3f6ff2d1f656f425a04e1cf8f8d693491260d7f0641f40774143965ea267a"}
Nov 04 03:22:56.297233 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.295489    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arwznk"]
Nov 04 03:22:56.299890 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.299354    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arwznk"
Nov 04 03:22:56.400080 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.396782    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"6541d93aaee335d6227a8778ab4e840bbe7f52-f98fbcf8-dockercfg-bw84h\""
Nov 04 03:22:56.400080 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.398324    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arwznk"]
Nov 04 03:22:56.400080 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.398383    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gdw75"]
Nov 04 03:22:56.402852 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.402280    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gdw75"
Nov 04 03:22:56.424578 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.424543    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5056bae9-a44b-4fa1-83ce-61c5736b3490-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arwznk\" (UID: \"5056bae9-a44b-4fa1-83ce-61c5736b3490\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arwznk"
Nov 04 03:22:56.424772 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.424711    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frgc8\" (UniqueName: \"kubernetes.io/projected/5056bae9-a44b-4fa1-83ce-61c5736b3490-kube-api-access-frgc8\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arwznk\" (UID: \"5056bae9-a44b-4fa1-83ce-61c5736b3490\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arwznk"
Nov 04 03:22:56.424839 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.424800    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5056bae9-a44b-4fa1-83ce-61c5736b3490-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arwznk\" (UID: \"5056bae9-a44b-4fa1-83ce-61c5736b3490\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arwznk"
Nov 04 03:22:56.452423 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.452337    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"da112bd609bff87f7bacd2ba941225d7cff15c-bd73b2fc-dockercfg-pgrkp\""
Nov 04 03:22:56.460663 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.460622    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gdw75"]
Nov 04 03:22:56.491085 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.491053    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c65smsf"]
Nov 04 03:22:56.494743 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.494710    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c65smsf"
Nov 04 03:22:56.525557 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.525524    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5056bae9-a44b-4fa1-83ce-61c5736b3490-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arwznk\" (UID: \"5056bae9-a44b-4fa1-83ce-61c5736b3490\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arwznk"
Nov 04 03:22:56.525747 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.525575    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5056bae9-a44b-4fa1-83ce-61c5736b3490-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arwznk\" (UID: \"5056bae9-a44b-4fa1-83ce-61c5736b3490\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arwznk"
Nov 04 03:22:56.525747 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.525636    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frgc8\" (UniqueName: \"kubernetes.io/projected/5056bae9-a44b-4fa1-83ce-61c5736b3490-kube-api-access-frgc8\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arwznk\" (UID: \"5056bae9-a44b-4fa1-83ce-61c5736b3490\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arwznk"
Nov 04 03:22:56.525747 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.525676    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a65be0cb-f19a-4303-9a56-f421a021f8eb-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gdw75\" (UID: \"a65be0cb-f19a-4303-9a56-f421a021f8eb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gdw75"
Nov 04 03:22:56.525747 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.525711    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8fkt\" (UniqueName: \"kubernetes.io/projected/a65be0cb-f19a-4303-9a56-f421a021f8eb-kube-api-access-s8fkt\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gdw75\" (UID: \"a65be0cb-f19a-4303-9a56-f421a021f8eb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gdw75"
Nov 04 03:22:56.525747 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.525741    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a65be0cb-f19a-4303-9a56-f421a021f8eb-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gdw75\" (UID: \"a65be0cb-f19a-4303-9a56-f421a021f8eb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gdw75"
Nov 04 03:22:56.525994 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.525977    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5056bae9-a44b-4fa1-83ce-61c5736b3490-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arwznk\" (UID: \"5056bae9-a44b-4fa1-83ce-61c5736b3490\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arwznk"
Nov 04 03:22:56.526069 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.526050    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5056bae9-a44b-4fa1-83ce-61c5736b3490-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arwznk\" (UID: \"5056bae9-a44b-4fa1-83ce-61c5736b3490\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arwznk"
Nov 04 03:22:56.558189 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.558092    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c65smsf"]
Nov 04 03:22:56.558492 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.558452    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3-b83c8cba-dockercfg-6ncz9\""
Nov 04 03:22:56.586274 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.586244    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frgc8\" (UniqueName: \"kubernetes.io/projected/5056bae9-a44b-4fa1-83ce-61c5736b3490-kube-api-access-frgc8\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arwznk\" (UID: \"5056bae9-a44b-4fa1-83ce-61c5736b3490\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arwznk"
Nov 04 03:22:56.592646 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.592150    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctn42v"]
Nov 04 03:22:56.595751 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.595734    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctn42v"
Nov 04 03:22:56.618555 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.618520    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arwznk"
Nov 04 03:22:56.627104 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.627077    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/11891172-56b2-42f2-b44d-c60dbccf4db8-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c65smsf\" (UID: \"11891172-56b2-42f2-b44d-c60dbccf4db8\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c65smsf"
Nov 04 03:22:56.627237 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.627136    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nscc\" (UniqueName: \"kubernetes.io/projected/11891172-56b2-42f2-b44d-c60dbccf4db8-kube-api-access-7nscc\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c65smsf\" (UID: \"11891172-56b2-42f2-b44d-c60dbccf4db8\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c65smsf"
Nov 04 03:22:56.627237 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.627220    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/11891172-56b2-42f2-b44d-c60dbccf4db8-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c65smsf\" (UID: \"11891172-56b2-42f2-b44d-c60dbccf4db8\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c65smsf"
Nov 04 03:22:56.627306 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.627263    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a65be0cb-f19a-4303-9a56-f421a021f8eb-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gdw75\" (UID: \"a65be0cb-f19a-4303-9a56-f421a021f8eb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gdw75"
Nov 04 03:22:56.627391 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.627313    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8fkt\" (UniqueName: \"kubernetes.io/projected/a65be0cb-f19a-4303-9a56-f421a021f8eb-kube-api-access-s8fkt\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gdw75\" (UID: \"a65be0cb-f19a-4303-9a56-f421a021f8eb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gdw75"
Nov 04 03:22:56.627391 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.627334    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a65be0cb-f19a-4303-9a56-f421a021f8eb-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gdw75\" (UID: \"a65be0cb-f19a-4303-9a56-f421a021f8eb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gdw75"
Nov 04 03:22:56.627700 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.627679    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a65be0cb-f19a-4303-9a56-f421a021f8eb-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gdw75\" (UID: \"a65be0cb-f19a-4303-9a56-f421a021f8eb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gdw75"
Nov 04 03:22:56.627777 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.627732    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a65be0cb-f19a-4303-9a56-f421a021f8eb-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gdw75\" (UID: \"a65be0cb-f19a-4303-9a56-f421a021f8eb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gdw75"
Nov 04 03:22:56.651449 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.651356    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctn42v"]
Nov 04 03:22:56.653048 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.653020    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e3d011e69d9573cabb8c359e5ac709eb5ac240-be78d160-dockercfg-nntwj\""
Nov 04 03:22:56.688341 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.688291    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8fkt\" (UniqueName: \"kubernetes.io/projected/a65be0cb-f19a-4303-9a56-f421a021f8eb-kube-api-access-s8fkt\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gdw75\" (UID: \"a65be0cb-f19a-4303-9a56-f421a021f8eb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gdw75"
Nov 04 03:22:56.724480 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.724440    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gdw75"
Nov 04 03:22:56.727732 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.727699    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/11891172-56b2-42f2-b44d-c60dbccf4db8-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c65smsf\" (UID: \"11891172-56b2-42f2-b44d-c60dbccf4db8\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c65smsf"
Nov 04 03:22:56.727859 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.727766    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b860e253-749d-4c29-8aa5-0dae76b4d78b-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctn42v\" (UID: \"b860e253-749d-4c29-8aa5-0dae76b4d78b\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctn42v"
Nov 04 03:22:56.727859 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.727801    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/11891172-56b2-42f2-b44d-c60dbccf4db8-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c65smsf\" (UID: \"11891172-56b2-42f2-b44d-c60dbccf4db8\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c65smsf"
Nov 04 03:22:56.727859 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.727838    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmtnx\" (UniqueName: \"kubernetes.io/projected/b860e253-749d-4c29-8aa5-0dae76b4d78b-kube-api-access-pmtnx\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctn42v\" (UID: \"b860e253-749d-4c29-8aa5-0dae76b4d78b\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctn42v"
Nov 04 03:22:56.728023 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.727868    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b860e253-749d-4c29-8aa5-0dae76b4d78b-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctn42v\" (UID: \"b860e253-749d-4c29-8aa5-0dae76b4d78b\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctn42v"
Nov 04 03:22:56.728023 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.727907    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nscc\" (UniqueName: \"kubernetes.io/projected/11891172-56b2-42f2-b44d-c60dbccf4db8-kube-api-access-7nscc\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c65smsf\" (UID: \"11891172-56b2-42f2-b44d-c60dbccf4db8\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c65smsf"
Nov 04 03:22:56.728783 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.728604    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/11891172-56b2-42f2-b44d-c60dbccf4db8-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c65smsf\" (UID: \"11891172-56b2-42f2-b44d-c60dbccf4db8\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c65smsf"
Nov 04 03:22:56.728783 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.728702    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/11891172-56b2-42f2-b44d-c60dbccf4db8-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c65smsf\" (UID: \"11891172-56b2-42f2-b44d-c60dbccf4db8\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c65smsf"
Nov 04 03:22:56.788577 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.787920    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nscc\" (UniqueName: \"kubernetes.io/projected/11891172-56b2-42f2-b44d-c60dbccf4db8-kube-api-access-7nscc\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c65smsf\" (UID: \"11891172-56b2-42f2-b44d-c60dbccf4db8\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c65smsf"
Nov 04 03:22:56.813578 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.813264    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c65smsf"
Nov 04 03:22:56.828847 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.828814    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b860e253-749d-4c29-8aa5-0dae76b4d78b-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctn42v\" (UID: \"b860e253-749d-4c29-8aa5-0dae76b4d78b\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctn42v"
Nov 04 03:22:56.828957 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.828892    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmtnx\" (UniqueName: \"kubernetes.io/projected/b860e253-749d-4c29-8aa5-0dae76b4d78b-kube-api-access-pmtnx\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctn42v\" (UID: \"b860e253-749d-4c29-8aa5-0dae76b4d78b\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctn42v"
Nov 04 03:22:56.828957 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.828928    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b860e253-749d-4c29-8aa5-0dae76b4d78b-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctn42v\" (UID: \"b860e253-749d-4c29-8aa5-0dae76b4d78b\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctn42v"
Nov 04 03:22:56.829702 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.829372    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b860e253-749d-4c29-8aa5-0dae76b4d78b-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctn42v\" (UID: \"b860e253-749d-4c29-8aa5-0dae76b4d78b\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctn42v"
Nov 04 03:22:56.829702 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.829379    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b860e253-749d-4c29-8aa5-0dae76b4d78b-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctn42v\" (UID: \"b860e253-749d-4c29-8aa5-0dae76b4d78b\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctn42v"
Nov 04 03:22:56.890122 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.889918    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmtnx\" (UniqueName: \"kubernetes.io/projected/b860e253-749d-4c29-8aa5-0dae76b4d78b-kube-api-access-pmtnx\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctn42v\" (UID: \"b860e253-749d-4c29-8aa5-0dae76b4d78b\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctn42v"
Nov 04 03:22:56.912891 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.912848    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctn42v"
Nov 04 03:22:56.999492 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:56.999453    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arwznk"]
Nov 04 03:22:57.006867 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:22:57.006821    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5056bae9_a44b_4fa1_83ce_61c5736b3490.slice/crio-4a5206d2c279c509d6ef297dfb2b1613fc2e6becb30fdcbb7eab5879e4491af6 WatchSource:0}: Error finding container 4a5206d2c279c509d6ef297dfb2b1613fc2e6becb30fdcbb7eab5879e4491af6: Status 404 returned error can't find the container with id 4a5206d2c279c509d6ef297dfb2b1613fc2e6becb30fdcbb7eab5879e4491af6
Nov 04 03:22:57.109552 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:57.109467    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gdw75"]
Nov 04 03:22:57.117111 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:22:57.117072    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda65be0cb_f19a_4303_9a56_f421a021f8eb.slice/crio-4a67a2aae5b05b97c1e79be169e6ebf76a98aac82daae646c251f5fde09e88a6 WatchSource:0}: Error finding container 4a67a2aae5b05b97c1e79be169e6ebf76a98aac82daae646c251f5fde09e88a6: Status 404 returned error can't find the container with id 4a67a2aae5b05b97c1e79be169e6ebf76a98aac82daae646c251f5fde09e88a6
Nov 04 03:22:57.193646 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:57.193574    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c65smsf"]
Nov 04 03:22:57.200516 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:22:57.200479    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11891172_56b2_42f2_b44d_c60dbccf4db8.slice/crio-56a8721217efc05848659d0fc6c8438fdfbd8d5b0d87afd347ff5f4ef0498249 WatchSource:0}: Error finding container 56a8721217efc05848659d0fc6c8438fdfbd8d5b0d87afd347ff5f4ef0498249: Status 404 returned error can't find the container with id 56a8721217efc05848659d0fc6c8438fdfbd8d5b0d87afd347ff5f4ef0498249
Nov 04 03:22:57.302090 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:57.302051    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctn42v"]
Nov 04 03:22:57.309709 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:22:57.309676    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb860e253_749d_4c29_8aa5_0dae76b4d78b.slice/crio-02e69e0ea114fb0c5c6aab99b6069702a1159e98ed4122f9ac42eea111e94413 WatchSource:0}: Error finding container 02e69e0ea114fb0c5c6aab99b6069702a1159e98ed4122f9ac42eea111e94413: Status 404 returned error can't find the container with id 02e69e0ea114fb0c5c6aab99b6069702a1159e98ed4122f9ac42eea111e94413
Nov 04 03:22:57.744022 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:57.743980    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctn42v" event={"ID":"b860e253-749d-4c29-8aa5-0dae76b4d78b","Type":"ContainerStarted","Data":"02e69e0ea114fb0c5c6aab99b6069702a1159e98ed4122f9ac42eea111e94413"}
Nov 04 03:22:57.745156 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:57.745125    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c65smsf" event={"ID":"11891172-56b2-42f2-b44d-c60dbccf4db8","Type":"ContainerStarted","Data":"56a8721217efc05848659d0fc6c8438fdfbd8d5b0d87afd347ff5f4ef0498249"}
Nov 04 03:22:57.746247 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:57.746218    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arwznk" event={"ID":"5056bae9-a44b-4fa1-83ce-61c5736b3490","Type":"ContainerStarted","Data":"4a5206d2c279c509d6ef297dfb2b1613fc2e6becb30fdcbb7eab5879e4491af6"}
Nov 04 03:22:57.747295 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:57.747272    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gdw75" event={"ID":"a65be0cb-f19a-4303-9a56-f421a021f8eb","Type":"ContainerStarted","Data":"4a67a2aae5b05b97c1e79be169e6ebf76a98aac82daae646c251f5fde09e88a6"}
Nov 04 03:22:59.574088 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:59.574048    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77"]
Nov 04 03:22:59.578541 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:59.577822    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77"
Nov 04 03:22:59.656005 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:59.655855    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Nov 04 03:22:59.656005 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:59.655973    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-dk4hf\""
Nov 04 03:22:59.658078 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:59.657553    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7flw\" (UniqueName: \"kubernetes.io/projected/8772b2ff-e4cc-4b41-85a9-491fbaf001c8-kube-api-access-j7flw\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-v5h77\" (UID: \"8772b2ff-e4cc-4b41-85a9-491fbaf001c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77"
Nov 04 03:22:59.658078 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:59.657637    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8772b2ff-e4cc-4b41-85a9-491fbaf001c8-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-v5h77\" (UID: \"8772b2ff-e4cc-4b41-85a9-491fbaf001c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77"
Nov 04 03:22:59.658078 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:59.657667    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8772b2ff-e4cc-4b41-85a9-491fbaf001c8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-v5h77\" (UID: \"8772b2ff-e4cc-4b41-85a9-491fbaf001c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77"
Nov 04 03:22:59.658078 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:59.657695    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8772b2ff-e4cc-4b41-85a9-491fbaf001c8-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-v5h77\" (UID: \"8772b2ff-e4cc-4b41-85a9-491fbaf001c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77"
Nov 04 03:22:59.658078 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:59.657726    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8772b2ff-e4cc-4b41-85a9-491fbaf001c8-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-v5h77\" (UID: \"8772b2ff-e4cc-4b41-85a9-491fbaf001c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77"
Nov 04 03:22:59.658078 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:59.657761    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8772b2ff-e4cc-4b41-85a9-491fbaf001c8-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-v5h77\" (UID: \"8772b2ff-e4cc-4b41-85a9-491fbaf001c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77"
Nov 04 03:22:59.658078 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:59.657792    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8772b2ff-e4cc-4b41-85a9-491fbaf001c8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-v5h77\" (UID: \"8772b2ff-e4cc-4b41-85a9-491fbaf001c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77"
Nov 04 03:22:59.658078 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:59.657822    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8772b2ff-e4cc-4b41-85a9-491fbaf001c8-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-v5h77\" (UID: \"8772b2ff-e4cc-4b41-85a9-491fbaf001c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77"
Nov 04 03:22:59.658078 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:59.657873    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8772b2ff-e4cc-4b41-85a9-491fbaf001c8-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-v5h77\" (UID: \"8772b2ff-e4cc-4b41-85a9-491fbaf001c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77"
Nov 04 03:22:59.686162 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:59.686088    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77"]
Nov 04 03:22:59.758619 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:59.758582    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8772b2ff-e4cc-4b41-85a9-491fbaf001c8-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-v5h77\" (UID: \"8772b2ff-e4cc-4b41-85a9-491fbaf001c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77"
Nov 04 03:22:59.758842 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:59.758640    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8772b2ff-e4cc-4b41-85a9-491fbaf001c8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-v5h77\" (UID: \"8772b2ff-e4cc-4b41-85a9-491fbaf001c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77"
Nov 04 03:22:59.758842 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:59.758674    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8772b2ff-e4cc-4b41-85a9-491fbaf001c8-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-v5h77\" (UID: \"8772b2ff-e4cc-4b41-85a9-491fbaf001c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77"
Nov 04 03:22:59.758842 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:59.758727    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8772b2ff-e4cc-4b41-85a9-491fbaf001c8-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-v5h77\" (UID: \"8772b2ff-e4cc-4b41-85a9-491fbaf001c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77"
Nov 04 03:22:59.758842 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:59.758756    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7flw\" (UniqueName: \"kubernetes.io/projected/8772b2ff-e4cc-4b41-85a9-491fbaf001c8-kube-api-access-j7flw\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-v5h77\" (UID: \"8772b2ff-e4cc-4b41-85a9-491fbaf001c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77"
Nov 04 03:22:59.759144 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:59.758880    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8772b2ff-e4cc-4b41-85a9-491fbaf001c8-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-v5h77\" (UID: \"8772b2ff-e4cc-4b41-85a9-491fbaf001c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77"
Nov 04 03:22:59.759144 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:59.758908    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8772b2ff-e4cc-4b41-85a9-491fbaf001c8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-v5h77\" (UID: \"8772b2ff-e4cc-4b41-85a9-491fbaf001c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77"
Nov 04 03:22:59.759144 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:59.758936    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8772b2ff-e4cc-4b41-85a9-491fbaf001c8-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-v5h77\" (UID: \"8772b2ff-e4cc-4b41-85a9-491fbaf001c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77"
Nov 04 03:22:59.759144 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:59.758969    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8772b2ff-e4cc-4b41-85a9-491fbaf001c8-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-v5h77\" (UID: \"8772b2ff-e4cc-4b41-85a9-491fbaf001c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77"
Nov 04 03:22:59.762476 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:59.762446    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8772b2ff-e4cc-4b41-85a9-491fbaf001c8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-v5h77\" (UID: \"8772b2ff-e4cc-4b41-85a9-491fbaf001c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77"
Nov 04 03:22:59.762769 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:59.762740    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8772b2ff-e4cc-4b41-85a9-491fbaf001c8-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-v5h77\" (UID: \"8772b2ff-e4cc-4b41-85a9-491fbaf001c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77"
Nov 04 03:22:59.763137 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:59.763115    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8772b2ff-e4cc-4b41-85a9-491fbaf001c8-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-v5h77\" (UID: \"8772b2ff-e4cc-4b41-85a9-491fbaf001c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77"
Nov 04 03:22:59.765796 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:59.765414    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8772b2ff-e4cc-4b41-85a9-491fbaf001c8-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-v5h77\" (UID: \"8772b2ff-e4cc-4b41-85a9-491fbaf001c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77"
Nov 04 03:22:59.765796 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:59.765747    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8772b2ff-e4cc-4b41-85a9-491fbaf001c8-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-v5h77\" (UID: \"8772b2ff-e4cc-4b41-85a9-491fbaf001c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77"
Nov 04 03:22:59.774751 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:59.774690    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8772b2ff-e4cc-4b41-85a9-491fbaf001c8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-v5h77\" (UID: \"8772b2ff-e4cc-4b41-85a9-491fbaf001c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77"
Nov 04 03:22:59.778749 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:59.778723    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8772b2ff-e4cc-4b41-85a9-491fbaf001c8-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-v5h77\" (UID: \"8772b2ff-e4cc-4b41-85a9-491fbaf001c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77"
Nov 04 03:22:59.821355 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:59.821292    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8772b2ff-e4cc-4b41-85a9-491fbaf001c8-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-v5h77\" (UID: \"8772b2ff-e4cc-4b41-85a9-491fbaf001c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77"
Nov 04 03:22:59.837810 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:59.837667    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7flw\" (UniqueName: \"kubernetes.io/projected/8772b2ff-e4cc-4b41-85a9-491fbaf001c8-kube-api-access-j7flw\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-v5h77\" (UID: \"8772b2ff-e4cc-4b41-85a9-491fbaf001c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77"
Nov 04 03:22:59.897059 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:59.896703    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77"
Nov 04 03:22:59.917732 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:59.917702    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8"]
Nov 04 03:22:59.921832 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:59.921757    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8"
Nov 04 03:22:59.972064 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:59.971638    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"openshift-ai-inference-openshift-ai-inference-dockercfg-pk6jv\""
Nov 04 03:22:59.985719 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:22:59.985316    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8"]
Nov 04 03:23:00.062929 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:00.062891    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc5b4561-6c05-4d20-8483-e4c83f3fef52-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8\" (UID: \"fc5b4561-6c05-4d20-8483-e4c83f3fef52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8"
Nov 04 03:23:00.063104 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:00.062950    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc5b4561-6c05-4d20-8483-e4c83f3fef52-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8\" (UID: \"fc5b4561-6c05-4d20-8483-e4c83f3fef52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8"
Nov 04 03:23:00.063104 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:00.062997    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc5b4561-6c05-4d20-8483-e4c83f3fef52-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8\" (UID: \"fc5b4561-6c05-4d20-8483-e4c83f3fef52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8"
Nov 04 03:23:00.063104 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:00.063021    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc5b4561-6c05-4d20-8483-e4c83f3fef52-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8\" (UID: \"fc5b4561-6c05-4d20-8483-e4c83f3fef52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8"
Nov 04 03:23:00.063104 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:00.063049    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc5b4561-6c05-4d20-8483-e4c83f3fef52-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8\" (UID: \"fc5b4561-6c05-4d20-8483-e4c83f3fef52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8"
Nov 04 03:23:00.063104 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:00.063102    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc5b4561-6c05-4d20-8483-e4c83f3fef52-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8\" (UID: \"fc5b4561-6c05-4d20-8483-e4c83f3fef52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8"
Nov 04 03:23:00.063312 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:00.063128    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2x4z\" (UniqueName: \"kubernetes.io/projected/fc5b4561-6c05-4d20-8483-e4c83f3fef52-kube-api-access-t2x4z\") pod \"openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8\" (UID: \"fc5b4561-6c05-4d20-8483-e4c83f3fef52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8"
Nov 04 03:23:00.063312 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:00.063162    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc5b4561-6c05-4d20-8483-e4c83f3fef52-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8\" (UID: \"fc5b4561-6c05-4d20-8483-e4c83f3fef52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8"
Nov 04 03:23:00.063312 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:00.063215    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc5b4561-6c05-4d20-8483-e4c83f3fef52-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8\" (UID: \"fc5b4561-6c05-4d20-8483-e4c83f3fef52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8"
Nov 04 03:23:00.164111 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:00.164030    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc5b4561-6c05-4d20-8483-e4c83f3fef52-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8\" (UID: \"fc5b4561-6c05-4d20-8483-e4c83f3fef52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8"
Nov 04 03:23:00.164111 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:00.164104    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc5b4561-6c05-4d20-8483-e4c83f3fef52-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8\" (UID: \"fc5b4561-6c05-4d20-8483-e4c83f3fef52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8"
Nov 04 03:23:00.164295 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:00.164137    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc5b4561-6c05-4d20-8483-e4c83f3fef52-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8\" (UID: \"fc5b4561-6c05-4d20-8483-e4c83f3fef52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8"
Nov 04 03:23:00.164295 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:00.164180    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc5b4561-6c05-4d20-8483-e4c83f3fef52-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8\" (UID: \"fc5b4561-6c05-4d20-8483-e4c83f3fef52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8"
Nov 04 03:23:00.164295 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:00.164205    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc5b4561-6c05-4d20-8483-e4c83f3fef52-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8\" (UID: \"fc5b4561-6c05-4d20-8483-e4c83f3fef52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8"
Nov 04 03:23:00.164295 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:00.164236    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc5b4561-6c05-4d20-8483-e4c83f3fef52-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8\" (UID: \"fc5b4561-6c05-4d20-8483-e4c83f3fef52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8"
Nov 04 03:23:00.164295 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:00.164288    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc5b4561-6c05-4d20-8483-e4c83f3fef52-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8\" (UID: \"fc5b4561-6c05-4d20-8483-e4c83f3fef52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8"
Nov 04 03:23:00.164459 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:00.164315    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2x4z\" (UniqueName: \"kubernetes.io/projected/fc5b4561-6c05-4d20-8483-e4c83f3fef52-kube-api-access-t2x4z\") pod \"openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8\" (UID: \"fc5b4561-6c05-4d20-8483-e4c83f3fef52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8"
Nov 04 03:23:00.164459 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:00.164349    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc5b4561-6c05-4d20-8483-e4c83f3fef52-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8\" (UID: \"fc5b4561-6c05-4d20-8483-e4c83f3fef52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8"
Nov 04 03:23:00.164882 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:00.164851    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc5b4561-6c05-4d20-8483-e4c83f3fef52-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8\" (UID: \"fc5b4561-6c05-4d20-8483-e4c83f3fef52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8"
Nov 04 03:23:00.167553 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:00.166113    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc5b4561-6c05-4d20-8483-e4c83f3fef52-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8\" (UID: \"fc5b4561-6c05-4d20-8483-e4c83f3fef52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8"
Nov 04 03:23:00.167553 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:00.166354    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc5b4561-6c05-4d20-8483-e4c83f3fef52-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8\" (UID: \"fc5b4561-6c05-4d20-8483-e4c83f3fef52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8"
Nov 04 03:23:00.167553 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:00.167216    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc5b4561-6c05-4d20-8483-e4c83f3fef52-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8\" (UID: \"fc5b4561-6c05-4d20-8483-e4c83f3fef52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8"
Nov 04 03:23:00.167553 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:00.167481    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc5b4561-6c05-4d20-8483-e4c83f3fef52-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8\" (UID: \"fc5b4561-6c05-4d20-8483-e4c83f3fef52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8"
Nov 04 03:23:00.168428 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:00.168311    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc5b4561-6c05-4d20-8483-e4c83f3fef52-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8\" (UID: \"fc5b4561-6c05-4d20-8483-e4c83f3fef52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8"
Nov 04 03:23:00.188463 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:00.188265    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc5b4561-6c05-4d20-8483-e4c83f3fef52-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8\" (UID: \"fc5b4561-6c05-4d20-8483-e4c83f3fef52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8"
Nov 04 03:23:00.225063 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:00.224985    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc5b4561-6c05-4d20-8483-e4c83f3fef52-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8\" (UID: \"fc5b4561-6c05-4d20-8483-e4c83f3fef52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8"
Nov 04 03:23:00.233671 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:00.233604    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2x4z\" (UniqueName: \"kubernetes.io/projected/fc5b4561-6c05-4d20-8483-e4c83f3fef52-kube-api-access-t2x4z\") pod \"openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8\" (UID: \"fc5b4561-6c05-4d20-8483-e4c83f3fef52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8"
Nov 04 03:23:00.252305 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:00.252273    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8"
Nov 04 03:23:00.396906 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:00.396803    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77"]
Nov 04 03:23:00.682735 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:00.682687    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8"]
Nov 04 03:23:00.700758 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:23:00.700727    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc5b4561_6c05_4d20_8483_e4c83f3fef52.slice/crio-f19cd6cc86d3337b1000bd244db567d1408d404e4c03a1e4fdb20c08504ad8c8 WatchSource:0}: Error finding container f19cd6cc86d3337b1000bd244db567d1408d404e4c03a1e4fdb20c08504ad8c8: Status 404 returned error can't find the container with id f19cd6cc86d3337b1000bd244db567d1408d404e4c03a1e4fdb20c08504ad8c8
Nov 04 03:23:00.773068 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:00.772974    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77" event={"ID":"8772b2ff-e4cc-4b41-85a9-491fbaf001c8","Type":"ContainerStarted","Data":"a39e0898836b121ac0ce9582ff4f65aac4b7544de810d294befccad8f120458d"}
Nov 04 03:23:00.775167 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:00.775099    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8" event={"ID":"fc5b4561-6c05-4d20-8483-e4c83f3fef52","Type":"ContainerStarted","Data":"f19cd6cc86d3337b1000bd244db567d1408d404e4c03a1e4fdb20c08504ad8c8"}
Nov 04 03:23:07.837732 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:07.837680    5609 generic.go:356] "Generic (PLEG): container finished" podID="c65ca1f2-8c4e-420d-8726-4d01a567a302" containerID="15129a0c01e3b45f50ee9f23b5c73b59272aefa9bcf10c614fc344ba281e746a" exitCode=0
Nov 04 03:23:07.838245 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:07.837852    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448d75f" event={"ID":"c65ca1f2-8c4e-420d-8726-4d01a567a302","Type":"ContainerDied","Data":"15129a0c01e3b45f50ee9f23b5c73b59272aefa9bcf10c614fc344ba281e746a"}
Nov 04 03:23:07.847395 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:07.847362    5609 generic.go:356] "Generic (PLEG): container finished" podID="11891172-56b2-42f2-b44d-c60dbccf4db8" containerID="c6f1b133985d695a6bf4853f207576aaffee2cdc0a3221dfac31ff77734583b0" exitCode=0
Nov 04 03:23:07.847523 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:07.847472    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c65smsf" event={"ID":"11891172-56b2-42f2-b44d-c60dbccf4db8","Type":"ContainerDied","Data":"c6f1b133985d695a6bf4853f207576aaffee2cdc0a3221dfac31ff77734583b0"}
Nov 04 03:23:08.859569 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:08.858476    5609 generic.go:356] "Generic (PLEG): container finished" podID="b860e253-749d-4c29-8aa5-0dae76b4d78b" containerID="aadf7274a3f5f7dfd3b922f704618aaf2486fe102d6cd8ebbd869684cfd457d7" exitCode=0
Nov 04 03:23:08.859569 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:08.858599    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctn42v" event={"ID":"b860e253-749d-4c29-8aa5-0dae76b4d78b","Type":"ContainerDied","Data":"aadf7274a3f5f7dfd3b922f704618aaf2486fe102d6cd8ebbd869684cfd457d7"}
Nov 04 03:23:09.416725 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:09.416655    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:23:09.416838 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:09.416796    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:23:09.416900 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:09.416835    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:23:09.466944 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:09.466903    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:23:09.467063 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:09.466981    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:23:09.467063 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:09.467039    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:23:09.870353 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:09.870132    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8" event={"ID":"fc5b4561-6c05-4d20-8483-e4c83f3fef52","Type":"ContainerStarted","Data":"44b25f8b544cbf5c02b8280bc3e918f44bf21a2a471965462124721b9e02169f"}
Nov 04 03:23:09.872496 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:09.872463    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77" event={"ID":"8772b2ff-e4cc-4b41-85a9-491fbaf001c8","Type":"ContainerStarted","Data":"ce2fe624e38171c11e491aa38095b48ba3c41cfd3675b824bef1436e712f344e"}
Nov 04 03:23:09.874366 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:09.874341    5609 generic.go:356] "Generic (PLEG): container finished" podID="c1a5a9af-8221-4655-ba83-7da2acf5244e" containerID="1a9a3069fa6b91a95b88dfef6dccf4d70063805689fcdb54b36c21bbe229939f" exitCode=0
Nov 04 03:23:09.874796 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:09.874772    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be712pstf" event={"ID":"c1a5a9af-8221-4655-ba83-7da2acf5244e","Type":"ContainerDied","Data":"1a9a3069fa6b91a95b88dfef6dccf4d70063805689fcdb54b36c21bbe229939f"}
Nov 04 03:23:09.877499 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:09.876793    5609 generic.go:356] "Generic (PLEG): container finished" podID="4558182a-d695-4e09-8956-60c86a9c02f1" containerID="cb6f9105e06a7a2c3c98d1a6ebef7dadbffe5874213d424f017a38368d4d76e3" exitCode=0
Nov 04 03:23:09.877499 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:09.876871    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cf7j9g" event={"ID":"4558182a-d695-4e09-8956-60c86a9c02f1","Type":"ContainerDied","Data":"cb6f9105e06a7a2c3c98d1a6ebef7dadbffe5874213d424f017a38368d4d76e3"}
Nov 04 03:23:09.884880 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:09.884366    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c65smsf" event={"ID":"11891172-56b2-42f2-b44d-c60dbccf4db8","Type":"ContainerStarted","Data":"0791f973861346c9e645f84102a77cdfe2e6d5bf1d98cb418f953d9c765914cf"}
Nov 04 03:23:09.897707 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:09.897681    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77"
Nov 04 03:23:10.100569 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:10.100507    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8" podStartSLOduration=2.389170551 podStartE2EDuration="11.100485284s" totalImagesPullingTime="8.711314733s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:22:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:23:09.986501675 +0000 UTC m=+332.563154602" watchObservedRunningTime="2026-11-04 03:23:10.100485284 +0000 UTC m=+332.677138199"
Nov 04 03:23:10.253166 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:10.253129    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8"
Nov 04 03:23:10.259054 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:10.259024    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8"
Nov 04 03:23:10.327259 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:10.327198    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77" podStartSLOduration=2.27379344 podStartE2EDuration="11.327180935s" totalImagesPullingTime="9.053387495s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:22:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:23:10.21548032 +0000 UTC m=+332.792133236" watchObservedRunningTime="2026-11-04 03:23:10.327180935 +0000 UTC m=+332.903833851"
Nov 04 03:23:10.891737 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:10.891695    5609 generic.go:356] "Generic (PLEG): container finished" podID="b860e253-749d-4c29-8aa5-0dae76b4d78b" containerID="ec67b4c5a3e061969e359dd27604f65360d78a57bd93dbc6b768f6efb2bcd62e" exitCode=0
Nov 04 03:23:10.892249 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:10.891774    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctn42v" event={"ID":"b860e253-749d-4c29-8aa5-0dae76b4d78b","Type":"ContainerDied","Data":"ec67b4c5a3e061969e359dd27604f65360d78a57bd93dbc6b768f6efb2bcd62e"}
Nov 04 03:23:10.894196 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:10.894166    5609 generic.go:356] "Generic (PLEG): container finished" podID="11891172-56b2-42f2-b44d-c60dbccf4db8" containerID="0791f973861346c9e645f84102a77cdfe2e6d5bf1d98cb418f953d9c765914cf" exitCode=0
Nov 04 03:23:10.894328 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:10.894193    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c65smsf" event={"ID":"11891172-56b2-42f2-b44d-c60dbccf4db8","Type":"ContainerDied","Data":"0791f973861346c9e645f84102a77cdfe2e6d5bf1d98cb418f953d9c765914cf"}
Nov 04 03:23:10.894742 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:10.894720    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8"
Nov 04 03:23:10.895995 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:10.895973    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8"
Nov 04 03:23:10.903529 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:10.903502    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77"
Nov 04 03:23:11.902430 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:11.901935    5609 generic.go:356] "Generic (PLEG): container finished" podID="5056bae9-a44b-4fa1-83ce-61c5736b3490" containerID="d9c75f57b7ce6d776c899f111a1cb8d0f166f547f96a15f692d0291fa5024502" exitCode=0
Nov 04 03:23:11.902430 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:11.902028    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arwznk" event={"ID":"5056bae9-a44b-4fa1-83ce-61c5736b3490","Type":"ContainerDied","Data":"d9c75f57b7ce6d776c899f111a1cb8d0f166f547f96a15f692d0291fa5024502"}
Nov 04 03:23:11.904111 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:11.904076    5609 generic.go:356] "Generic (PLEG): container finished" podID="a65be0cb-f19a-4303-9a56-f421a021f8eb" containerID="b7dbae4d5fa19764c6a34a5db8f5bb6cd83eaa4aefe341a753e8dd30991d0931" exitCode=0
Nov 04 03:23:11.904196 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:11.904165    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gdw75" event={"ID":"a65be0cb-f19a-4303-9a56-f421a021f8eb","Type":"ContainerDied","Data":"b7dbae4d5fa19764c6a34a5db8f5bb6cd83eaa4aefe341a753e8dd30991d0931"}
Nov 04 03:23:11.906554 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:11.906527    5609 generic.go:356] "Generic (PLEG): container finished" podID="c1a5a9af-8221-4655-ba83-7da2acf5244e" containerID="0061bcfd5d0978cbe2c5ceaf95858cb0173c657cd6a4de528620ac13ea347807" exitCode=0
Nov 04 03:23:11.906634 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:11.906610    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be712pstf" event={"ID":"c1a5a9af-8221-4655-ba83-7da2acf5244e","Type":"ContainerDied","Data":"0061bcfd5d0978cbe2c5ceaf95858cb0173c657cd6a4de528620ac13ea347807"}
Nov 04 03:23:11.907116 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:11.907097    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77"
Nov 04 03:23:11.908565 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:11.908485    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77"
Nov 04 03:23:13.448737 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:13.448680    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m7vcx"]
Nov 04 03:23:13.453566 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:13.453540    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m7vcx"
Nov 04 03:23:13.507204 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:13.507172    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m7vcx"]
Nov 04 03:23:13.602666 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:13.602445    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gvsv\" (UniqueName: \"kubernetes.io/projected/07ece136-c4aa-4998-be0f-0099f6f1e590-kube-api-access-6gvsv\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m7vcx\" (UID: \"07ece136-c4aa-4998-be0f-0099f6f1e590\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m7vcx"
Nov 04 03:23:13.602666 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:13.602492    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/07ece136-c4aa-4998-be0f-0099f6f1e590-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m7vcx\" (UID: \"07ece136-c4aa-4998-be0f-0099f6f1e590\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m7vcx"
Nov 04 03:23:13.602666 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:13.602516    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/07ece136-c4aa-4998-be0f-0099f6f1e590-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m7vcx\" (UID: \"07ece136-c4aa-4998-be0f-0099f6f1e590\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m7vcx"
Nov 04 03:23:13.618434 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:13.618044    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k7w5r"]
Nov 04 03:23:13.623541 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:13.622631    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k7w5r"
Nov 04 03:23:13.681767 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:13.681689    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k7w5r"]
Nov 04 03:23:13.703202 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:13.703167    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/beae230e-3fd0-41db-bcc8-8ec138d2c3db-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k7w5r\" (UID: \"beae230e-3fd0-41db-bcc8-8ec138d2c3db\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k7w5r"
Nov 04 03:23:13.703379 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:13.703213    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/beae230e-3fd0-41db-bcc8-8ec138d2c3db-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k7w5r\" (UID: \"beae230e-3fd0-41db-bcc8-8ec138d2c3db\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k7w5r"
Nov 04 03:23:13.703379 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:13.703252    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gvsv\" (UniqueName: \"kubernetes.io/projected/07ece136-c4aa-4998-be0f-0099f6f1e590-kube-api-access-6gvsv\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m7vcx\" (UID: \"07ece136-c4aa-4998-be0f-0099f6f1e590\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m7vcx"
Nov 04 03:23:13.703379 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:13.703298    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/07ece136-c4aa-4998-be0f-0099f6f1e590-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m7vcx\" (UID: \"07ece136-c4aa-4998-be0f-0099f6f1e590\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m7vcx"
Nov 04 03:23:13.703379 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:13.703328    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z29p4\" (UniqueName: \"kubernetes.io/projected/beae230e-3fd0-41db-bcc8-8ec138d2c3db-kube-api-access-z29p4\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k7w5r\" (UID: \"beae230e-3fd0-41db-bcc8-8ec138d2c3db\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k7w5r"
Nov 04 03:23:13.703379 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:13.703354    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/07ece136-c4aa-4998-be0f-0099f6f1e590-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m7vcx\" (UID: \"07ece136-c4aa-4998-be0f-0099f6f1e590\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m7vcx"
Nov 04 03:23:13.704486 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:13.703901    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/07ece136-c4aa-4998-be0f-0099f6f1e590-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m7vcx\" (UID: \"07ece136-c4aa-4998-be0f-0099f6f1e590\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m7vcx"
Nov 04 03:23:13.704486 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:13.704446    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/07ece136-c4aa-4998-be0f-0099f6f1e590-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m7vcx\" (UID: \"07ece136-c4aa-4998-be0f-0099f6f1e590\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m7vcx"
Nov 04 03:23:13.766705 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:13.766673    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gvsv\" (UniqueName: \"kubernetes.io/projected/07ece136-c4aa-4998-be0f-0099f6f1e590-kube-api-access-6gvsv\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m7vcx\" (UID: \"07ece136-c4aa-4998-be0f-0099f6f1e590\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m7vcx"
Nov 04 03:23:13.777056 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:13.776859    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m7vcx"
Nov 04 03:23:13.804057 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:13.804023    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/beae230e-3fd0-41db-bcc8-8ec138d2c3db-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k7w5r\" (UID: \"beae230e-3fd0-41db-bcc8-8ec138d2c3db\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k7w5r"
Nov 04 03:23:13.804057 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:13.804076    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/beae230e-3fd0-41db-bcc8-8ec138d2c3db-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k7w5r\" (UID: \"beae230e-3fd0-41db-bcc8-8ec138d2c3db\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k7w5r"
Nov 04 03:23:13.804057 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:13.804134    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z29p4\" (UniqueName: \"kubernetes.io/projected/beae230e-3fd0-41db-bcc8-8ec138d2c3db-kube-api-access-z29p4\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k7w5r\" (UID: \"beae230e-3fd0-41db-bcc8-8ec138d2c3db\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k7w5r"
Nov 04 03:23:13.804057 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:13.804900    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/beae230e-3fd0-41db-bcc8-8ec138d2c3db-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k7w5r\" (UID: \"beae230e-3fd0-41db-bcc8-8ec138d2c3db\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k7w5r"
Nov 04 03:23:13.805509 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:13.805468    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/beae230e-3fd0-41db-bcc8-8ec138d2c3db-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k7w5r\" (UID: \"beae230e-3fd0-41db-bcc8-8ec138d2c3db\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k7w5r"
Nov 04 03:23:13.863740 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:13.863680    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z29p4\" (UniqueName: \"kubernetes.io/projected/beae230e-3fd0-41db-bcc8-8ec138d2c3db-kube-api-access-z29p4\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k7w5r\" (UID: \"beae230e-3fd0-41db-bcc8-8ec138d2c3db\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k7w5r"
Nov 04 03:23:13.923412 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:13.923360    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arwznk" event={"ID":"5056bae9-a44b-4fa1-83ce-61c5736b3490","Type":"ContainerStarted","Data":"40cb8c910329d31c19ccfb318c948095dbf4cbeef7e953a12402801b24890119"}
Nov 04 03:23:13.926380 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:13.926345    5609 generic.go:356] "Generic (PLEG): container finished" podID="c65ca1f2-8c4e-420d-8726-4d01a567a302" containerID="857cedc161bebabebc6db744cdb1a9561513fa65102ef8b615e073a92f716362" exitCode=0
Nov 04 03:23:13.926529 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:13.926448    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448d75f" event={"ID":"c65ca1f2-8c4e-420d-8726-4d01a567a302","Type":"ContainerDied","Data":"857cedc161bebabebc6db744cdb1a9561513fa65102ef8b615e073a92f716362"}
Nov 04 03:23:13.930600 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:13.929690    5609 generic.go:356] "Generic (PLEG): container finished" podID="a65be0cb-f19a-4303-9a56-f421a021f8eb" containerID="e7253cd085ffd836a730ca2af9068f9d8e5b1e0958e7b9cf5700b09462280968" exitCode=0
Nov 04 03:23:13.930600 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:13.930509    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gdw75" event={"ID":"a65be0cb-f19a-4303-9a56-f421a021f8eb","Type":"ContainerDied","Data":"e7253cd085ffd836a730ca2af9068f9d8e5b1e0958e7b9cf5700b09462280968"}
Nov 04 03:23:13.944773 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:13.944534    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k7w5r"
Nov 04 03:23:14.177886 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:14.177824    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m7vcx"]
Nov 04 03:23:14.186989 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:23:14.186955    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07ece136_c4aa_4998_be0f_0099f6f1e590.slice/crio-86ff8af63a7c2fd7b245076cf5791a39296c48f095e41393cd0ac59ceee8fe67 WatchSource:0}: Error finding container 86ff8af63a7c2fd7b245076cf5791a39296c48f095e41393cd0ac59ceee8fe67: Status 404 returned error can't find the container with id 86ff8af63a7c2fd7b245076cf5791a39296c48f095e41393cd0ac59ceee8fe67
Nov 04 03:23:14.354800 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:14.354763    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k7w5r"]
Nov 04 03:23:14.938659 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:14.938369    5609 generic.go:356] "Generic (PLEG): container finished" podID="5056bae9-a44b-4fa1-83ce-61c5736b3490" containerID="40cb8c910329d31c19ccfb318c948095dbf4cbeef7e953a12402801b24890119" exitCode=0
Nov 04 03:23:14.939267 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:14.939222    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arwznk" event={"ID":"5056bae9-a44b-4fa1-83ce-61c5736b3490","Type":"ContainerDied","Data":"40cb8c910329d31c19ccfb318c948095dbf4cbeef7e953a12402801b24890119"}
Nov 04 03:23:14.942426 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:14.941689    5609 generic.go:356] "Generic (PLEG): container finished" podID="07ece136-c4aa-4998-be0f-0099f6f1e590" containerID="ddf3108313bcb19cc08577e8b28dd8debeb45ab1d0298a85db354cae7f6e3aeb" exitCode=0
Nov 04 03:23:14.942426 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:14.941824    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m7vcx" event={"ID":"07ece136-c4aa-4998-be0f-0099f6f1e590","Type":"ContainerDied","Data":"ddf3108313bcb19cc08577e8b28dd8debeb45ab1d0298a85db354cae7f6e3aeb"}
Nov 04 03:23:14.942426 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:14.941862    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m7vcx" event={"ID":"07ece136-c4aa-4998-be0f-0099f6f1e590","Type":"ContainerStarted","Data":"86ff8af63a7c2fd7b245076cf5791a39296c48f095e41393cd0ac59ceee8fe67"}
Nov 04 03:23:14.947428 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:14.946612    5609 generic.go:356] "Generic (PLEG): container finished" podID="4558182a-d695-4e09-8956-60c86a9c02f1" containerID="1bc7dc6be49e99baa156853f4262a92b7373e51dbe57aa3a3a72fb037758f83b" exitCode=0
Nov 04 03:23:14.947428 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:14.946737    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cf7j9g" event={"ID":"4558182a-d695-4e09-8956-60c86a9c02f1","Type":"ContainerDied","Data":"1bc7dc6be49e99baa156853f4262a92b7373e51dbe57aa3a3a72fb037758f83b"}
Nov 04 03:23:14.956065 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:14.955384    5609 generic.go:356] "Generic (PLEG): container finished" podID="beae230e-3fd0-41db-bcc8-8ec138d2c3db" containerID="e6f39c373bf66829f76281a3d33d8f6fec05f87c3e507862ec73cf7616ab453c" exitCode=0
Nov 04 03:23:14.956065 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:14.955433    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k7w5r" event={"ID":"beae230e-3fd0-41db-bcc8-8ec138d2c3db","Type":"ContainerDied","Data":"e6f39c373bf66829f76281a3d33d8f6fec05f87c3e507862ec73cf7616ab453c"}
Nov 04 03:23:14.956065 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:14.955473    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k7w5r" event={"ID":"beae230e-3fd0-41db-bcc8-8ec138d2c3db","Type":"ContainerStarted","Data":"f57e90ee9a7d9e1e5a5e6bd08fe86efa922572d464cd0c6b4af6d9b10f89a90e"}
Nov 04 03:23:15.242679 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:15.241545    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvvl4h"]
Nov 04 03:23:15.275161 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:15.275128    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvvl4h"
Nov 04 03:23:15.301361 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:15.301328    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvvl4h"]
Nov 04 03:23:15.424082 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:15.424041    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c0753e0-3eda-4c9b-a763-ca1c20982fed-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvvl4h\" (UID: \"7c0753e0-3eda-4c9b-a763-ca1c20982fed\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvvl4h"
Nov 04 03:23:15.424276 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:15.424179    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-485qt\" (UniqueName: \"kubernetes.io/projected/7c0753e0-3eda-4c9b-a763-ca1c20982fed-kube-api-access-485qt\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvvl4h\" (UID: \"7c0753e0-3eda-4c9b-a763-ca1c20982fed\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvvl4h"
Nov 04 03:23:15.424276 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:15.424249    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c0753e0-3eda-4c9b-a763-ca1c20982fed-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvvl4h\" (UID: \"7c0753e0-3eda-4c9b-a763-ca1c20982fed\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvvl4h"
Nov 04 03:23:15.525055 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:15.524972    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c0753e0-3eda-4c9b-a763-ca1c20982fed-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvvl4h\" (UID: \"7c0753e0-3eda-4c9b-a763-ca1c20982fed\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvvl4h"
Nov 04 03:23:15.525207 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:15.525059    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-485qt\" (UniqueName: \"kubernetes.io/projected/7c0753e0-3eda-4c9b-a763-ca1c20982fed-kube-api-access-485qt\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvvl4h\" (UID: \"7c0753e0-3eda-4c9b-a763-ca1c20982fed\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvvl4h"
Nov 04 03:23:15.525207 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:15.525149    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c0753e0-3eda-4c9b-a763-ca1c20982fed-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvvl4h\" (UID: \"7c0753e0-3eda-4c9b-a763-ca1c20982fed\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvvl4h"
Nov 04 03:23:15.526579 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:15.525678    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c0753e0-3eda-4c9b-a763-ca1c20982fed-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvvl4h\" (UID: \"7c0753e0-3eda-4c9b-a763-ca1c20982fed\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvvl4h"
Nov 04 03:23:15.526579 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:15.525959    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c0753e0-3eda-4c9b-a763-ca1c20982fed-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvvl4h\" (UID: \"7c0753e0-3eda-4c9b-a763-ca1c20982fed\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvvl4h"
Nov 04 03:23:15.586103 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:15.586067    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-485qt\" (UniqueName: \"kubernetes.io/projected/7c0753e0-3eda-4c9b-a763-ca1c20982fed-kube-api-access-485qt\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvvl4h\" (UID: \"7c0753e0-3eda-4c9b-a763-ca1c20982fed\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvvl4h"
Nov 04 03:23:15.611764 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:15.611722    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvvl4h"
Nov 04 03:23:15.969420 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:15.969364    5609 generic.go:356] "Generic (PLEG): container finished" podID="beae230e-3fd0-41db-bcc8-8ec138d2c3db" containerID="aec4f5c2f6a1ea0235d9009b78bc51af659dd05140daa7585a18584398890bef" exitCode=0
Nov 04 03:23:15.969895 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:15.969464    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k7w5r" event={"ID":"beae230e-3fd0-41db-bcc8-8ec138d2c3db","Type":"ContainerDied","Data":"aec4f5c2f6a1ea0235d9009b78bc51af659dd05140daa7585a18584398890bef"}
Nov 04 03:23:16.039593 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:16.039562    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvvl4h"]
Nov 04 03:23:16.051191 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:23:16.051160    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c0753e0_3eda_4c9b_a763_ca1c20982fed.slice/crio-4a31a2e7d30235b7f088d2ac5c06f93fb17757ffa47a7a012cb037110156c2d3 WatchSource:0}: Error finding container 4a31a2e7d30235b7f088d2ac5c06f93fb17757ffa47a7a012cb037110156c2d3: Status 404 returned error can't find the container with id 4a31a2e7d30235b7f088d2ac5c06f93fb17757ffa47a7a012cb037110156c2d3
Nov 04 03:23:16.723454 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:16.721679    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c689lh"]
Nov 04 03:23:16.759495 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:16.757925    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64k9b5"]
Nov 04 03:23:16.759875 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:16.759847    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c689lh"
Nov 04 03:23:16.788783 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:16.788065    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c689lh"]
Nov 04 03:23:16.788783 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:16.788116    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9tp7b"]
Nov 04 03:23:16.788783 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:16.788169    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64k9b5"
Nov 04 03:23:16.821419 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:16.821332    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pvbww"]
Nov 04 03:23:16.821571 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:16.821481    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9tp7b"
Nov 04 03:23:16.863467 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:16.863430    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64k9b5"]
Nov 04 03:23:16.863632 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:16.863534    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pvbww"
Nov 04 03:23:16.867374 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:16.867345    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9tp7b"]
Nov 04 03:23:16.886689 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:16.886660    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pvbww"]
Nov 04 03:23:16.941892 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:16.941308    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fb6vl\" (UniqueName: \"kubernetes.io/projected/179b6ae6-dd9d-4a69-8796-a1016dc2fef0-kube-api-access-fb6vl\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9tp7b\" (UID: \"179b6ae6-dd9d-4a69-8796-a1016dc2fef0\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9tp7b"
Nov 04 03:23:16.941892 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:16.941372    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/76d623cd-5534-4184-ae5e-de00ee4bc6c6-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64k9b5\" (UID: \"76d623cd-5534-4184-ae5e-de00ee4bc6c6\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64k9b5"
Nov 04 03:23:16.941892 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:16.941424    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/76d623cd-5534-4184-ae5e-de00ee4bc6c6-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64k9b5\" (UID: \"76d623cd-5534-4184-ae5e-de00ee4bc6c6\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64k9b5"
Nov 04 03:23:16.941892 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:16.941519    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9n2j\" (UniqueName: \"kubernetes.io/projected/76d623cd-5534-4184-ae5e-de00ee4bc6c6-kube-api-access-k9n2j\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64k9b5\" (UID: \"76d623cd-5534-4184-ae5e-de00ee4bc6c6\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64k9b5"
Nov 04 03:23:16.941892 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:16.941549    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spgcv\" (UniqueName: \"kubernetes.io/projected/993ee084-01db-4f9e-8f56-fd3bd5a407a3-kube-api-access-spgcv\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c689lh\" (UID: \"993ee084-01db-4f9e-8f56-fd3bd5a407a3\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c689lh"
Nov 04 03:23:16.941892 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:16.941609    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/993ee084-01db-4f9e-8f56-fd3bd5a407a3-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c689lh\" (UID: \"993ee084-01db-4f9e-8f56-fd3bd5a407a3\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c689lh"
Nov 04 03:23:16.941892 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:16.941638    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/179b6ae6-dd9d-4a69-8796-a1016dc2fef0-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9tp7b\" (UID: \"179b6ae6-dd9d-4a69-8796-a1016dc2fef0\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9tp7b"
Nov 04 03:23:16.941892 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:16.941667    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/179b6ae6-dd9d-4a69-8796-a1016dc2fef0-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9tp7b\" (UID: \"179b6ae6-dd9d-4a69-8796-a1016dc2fef0\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9tp7b"
Nov 04 03:23:16.941892 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:16.941704    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/993ee084-01db-4f9e-8f56-fd3bd5a407a3-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c689lh\" (UID: \"993ee084-01db-4f9e-8f56-fd3bd5a407a3\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c689lh"
Nov 04 03:23:16.999507 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:16.998628    5609 generic.go:356] "Generic (PLEG): container finished" podID="7c0753e0-3eda-4c9b-a763-ca1c20982fed" containerID="0638741c832824fa98a70580ca1b3e93ead8f4e204c544bba5a843f3b83ae814" exitCode=0
Nov 04 03:23:16.999965 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:16.999508    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvvl4h" event={"ID":"7c0753e0-3eda-4c9b-a763-ca1c20982fed","Type":"ContainerDied","Data":"0638741c832824fa98a70580ca1b3e93ead8f4e204c544bba5a843f3b83ae814"}
Nov 04 03:23:16.999965 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:16.999597    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvvl4h" event={"ID":"7c0753e0-3eda-4c9b-a763-ca1c20982fed","Type":"ContainerStarted","Data":"4a31a2e7d30235b7f088d2ac5c06f93fb17757ffa47a7a012cb037110156c2d3"}
Nov 04 03:23:17.043676 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:17.043046    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/993ee084-01db-4f9e-8f56-fd3bd5a407a3-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c689lh\" (UID: \"993ee084-01db-4f9e-8f56-fd3bd5a407a3\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c689lh"
Nov 04 03:23:17.043676 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:17.043535    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/993ee084-01db-4f9e-8f56-fd3bd5a407a3-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c689lh\" (UID: \"993ee084-01db-4f9e-8f56-fd3bd5a407a3\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c689lh"
Nov 04 03:23:17.045566 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:17.043098    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/184d53fa-c195-4625-bb1d-9310f983d9fb-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pvbww\" (UID: \"184d53fa-c195-4625-bb1d-9310f983d9fb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pvbww"
Nov 04 03:23:17.045566 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:17.044104    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/179b6ae6-dd9d-4a69-8796-a1016dc2fef0-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9tp7b\" (UID: \"179b6ae6-dd9d-4a69-8796-a1016dc2fef0\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9tp7b"
Nov 04 03:23:17.045566 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:17.044136    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/179b6ae6-dd9d-4a69-8796-a1016dc2fef0-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9tp7b\" (UID: \"179b6ae6-dd9d-4a69-8796-a1016dc2fef0\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9tp7b"
Nov 04 03:23:17.045566 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:17.044178    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/993ee084-01db-4f9e-8f56-fd3bd5a407a3-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c689lh\" (UID: \"993ee084-01db-4f9e-8f56-fd3bd5a407a3\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c689lh"
Nov 04 03:23:17.045566 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:17.044217    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fb6vl\" (UniqueName: \"kubernetes.io/projected/179b6ae6-dd9d-4a69-8796-a1016dc2fef0-kube-api-access-fb6vl\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9tp7b\" (UID: \"179b6ae6-dd9d-4a69-8796-a1016dc2fef0\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9tp7b"
Nov 04 03:23:17.045566 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:17.044261    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/76d623cd-5534-4184-ae5e-de00ee4bc6c6-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64k9b5\" (UID: \"76d623cd-5534-4184-ae5e-de00ee4bc6c6\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64k9b5"
Nov 04 03:23:17.045566 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:17.044288    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/76d623cd-5534-4184-ae5e-de00ee4bc6c6-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64k9b5\" (UID: \"76d623cd-5534-4184-ae5e-de00ee4bc6c6\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64k9b5"
Nov 04 03:23:17.045566 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:17.044332    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/184d53fa-c195-4625-bb1d-9310f983d9fb-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pvbww\" (UID: \"184d53fa-c195-4625-bb1d-9310f983d9fb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pvbww"
Nov 04 03:23:17.045566 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:17.044807    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/76d623cd-5534-4184-ae5e-de00ee4bc6c6-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64k9b5\" (UID: \"76d623cd-5534-4184-ae5e-de00ee4bc6c6\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64k9b5"
Nov 04 03:23:17.045566 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:17.045094    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/76d623cd-5534-4184-ae5e-de00ee4bc6c6-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64k9b5\" (UID: \"76d623cd-5534-4184-ae5e-de00ee4bc6c6\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64k9b5"
Nov 04 03:23:17.045566 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:17.045478    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/179b6ae6-dd9d-4a69-8796-a1016dc2fef0-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9tp7b\" (UID: \"179b6ae6-dd9d-4a69-8796-a1016dc2fef0\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9tp7b"
Nov 04 03:23:17.046159 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:17.045715    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/179b6ae6-dd9d-4a69-8796-a1016dc2fef0-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9tp7b\" (UID: \"179b6ae6-dd9d-4a69-8796-a1016dc2fef0\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9tp7b"
Nov 04 03:23:17.046159 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:17.045951    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/993ee084-01db-4f9e-8f56-fd3bd5a407a3-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c689lh\" (UID: \"993ee084-01db-4f9e-8f56-fd3bd5a407a3\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c689lh"
Nov 04 03:23:17.046792 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:17.046313    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmjfr\" (UniqueName: \"kubernetes.io/projected/184d53fa-c195-4625-bb1d-9310f983d9fb-kube-api-access-lmjfr\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pvbww\" (UID: \"184d53fa-c195-4625-bb1d-9310f983d9fb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pvbww"
Nov 04 03:23:17.046792 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:17.046369    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9n2j\" (UniqueName: \"kubernetes.io/projected/76d623cd-5534-4184-ae5e-de00ee4bc6c6-kube-api-access-k9n2j\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64k9b5\" (UID: \"76d623cd-5534-4184-ae5e-de00ee4bc6c6\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64k9b5"
Nov 04 03:23:17.046792 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:17.046415    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spgcv\" (UniqueName: \"kubernetes.io/projected/993ee084-01db-4f9e-8f56-fd3bd5a407a3-kube-api-access-spgcv\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c689lh\" (UID: \"993ee084-01db-4f9e-8f56-fd3bd5a407a3\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c689lh"
Nov 04 03:23:17.055338 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:17.054536    5609 generic.go:356] "Generic (PLEG): container finished" podID="07ece136-c4aa-4998-be0f-0099f6f1e590" containerID="862ec262625e73c404706985da31e63b7c89daa16353835919f06e57e53b6dde" exitCode=0
Nov 04 03:23:17.055338 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:17.054640    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m7vcx" event={"ID":"07ece136-c4aa-4998-be0f-0099f6f1e590","Type":"ContainerDied","Data":"862ec262625e73c404706985da31e63b7c89daa16353835919f06e57e53b6dde"}
Nov 04 03:23:17.107142 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:17.107113    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-spgcv\" (UniqueName: \"kubernetes.io/projected/993ee084-01db-4f9e-8f56-fd3bd5a407a3-kube-api-access-spgcv\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c689lh\" (UID: \"993ee084-01db-4f9e-8f56-fd3bd5a407a3\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c689lh"
Nov 04 03:23:17.111446 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:17.111341    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fb6vl\" (UniqueName: \"kubernetes.io/projected/179b6ae6-dd9d-4a69-8796-a1016dc2fef0-kube-api-access-fb6vl\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9tp7b\" (UID: \"179b6ae6-dd9d-4a69-8796-a1016dc2fef0\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9tp7b"
Nov 04 03:23:17.113927 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:17.113866    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9n2j\" (UniqueName: \"kubernetes.io/projected/76d623cd-5534-4184-ae5e-de00ee4bc6c6-kube-api-access-k9n2j\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64k9b5\" (UID: \"76d623cd-5534-4184-ae5e-de00ee4bc6c6\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64k9b5"
Nov 04 03:23:17.149231 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:17.147699    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/184d53fa-c195-4625-bb1d-9310f983d9fb-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pvbww\" (UID: \"184d53fa-c195-4625-bb1d-9310f983d9fb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pvbww"
Nov 04 03:23:17.149231 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:17.147797    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/184d53fa-c195-4625-bb1d-9310f983d9fb-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pvbww\" (UID: \"184d53fa-c195-4625-bb1d-9310f983d9fb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pvbww"
Nov 04 03:23:17.149231 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:17.147859    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmjfr\" (UniqueName: \"kubernetes.io/projected/184d53fa-c195-4625-bb1d-9310f983d9fb-kube-api-access-lmjfr\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pvbww\" (UID: \"184d53fa-c195-4625-bb1d-9310f983d9fb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pvbww"
Nov 04 03:23:17.149231 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:17.148630    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/184d53fa-c195-4625-bb1d-9310f983d9fb-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pvbww\" (UID: \"184d53fa-c195-4625-bb1d-9310f983d9fb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pvbww"
Nov 04 03:23:17.149231 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:17.148930    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/184d53fa-c195-4625-bb1d-9310f983d9fb-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pvbww\" (UID: \"184d53fa-c195-4625-bb1d-9310f983d9fb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pvbww"
Nov 04 03:23:17.154008 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:17.153777    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c689lh"
Nov 04 03:23:17.210475 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:17.210410    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmjfr\" (UniqueName: \"kubernetes.io/projected/184d53fa-c195-4625-bb1d-9310f983d9fb-kube-api-access-lmjfr\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pvbww\" (UID: \"184d53fa-c195-4625-bb1d-9310f983d9fb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pvbww"
Nov 04 03:23:17.300035 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:17.299804    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64k9b5"
Nov 04 03:23:17.330810 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:17.330544    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9tp7b"
Nov 04 03:23:17.380978 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:17.380685    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pvbww"
Nov 04 03:23:17.589623 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:17.588324    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c689lh"]
Nov 04 03:23:17.600545 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:23:17.600494    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod993ee084_01db_4f9e_8f56_fd3bd5a407a3.slice/crio-ad1506fbc911ddf45247260e273a7fab2b75d76f6c78ad0efcfb653e38c130c1 WatchSource:0}: Error finding container ad1506fbc911ddf45247260e273a7fab2b75d76f6c78ad0efcfb653e38c130c1: Status 404 returned error can't find the container with id ad1506fbc911ddf45247260e273a7fab2b75d76f6c78ad0efcfb653e38c130c1
Nov 04 03:23:17.760505 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:17.760472    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64k9b5"]
Nov 04 03:23:17.777549 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:23:17.777350    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76d623cd_5534_4184_ae5e_de00ee4bc6c6.slice/crio-20cff31f6567026cd9404f99207e82cf5e57a873b1dc399557757928d61e2fa4 WatchSource:0}: Error finding container 20cff31f6567026cd9404f99207e82cf5e57a873b1dc399557757928d61e2fa4: Status 404 returned error can't find the container with id 20cff31f6567026cd9404f99207e82cf5e57a873b1dc399557757928d61e2fa4
Nov 04 03:23:17.808698 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:17.808671    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9tp7b"]
Nov 04 03:23:17.841187 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:23:17.841117    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod179b6ae6_dd9d_4a69_8796_a1016dc2fef0.slice/crio-cd31a844f93e484ea98fa63b4d4453565105179b8c5be7c51dc1ee0e31ddeec4 WatchSource:0}: Error finding container cd31a844f93e484ea98fa63b4d4453565105179b8c5be7c51dc1ee0e31ddeec4: Status 404 returned error can't find the container with id cd31a844f93e484ea98fa63b4d4453565105179b8c5be7c51dc1ee0e31ddeec4
Nov 04 03:23:17.872929 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:17.872887    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pvbww"]
Nov 04 03:23:17.893944 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:23:17.893697    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod184d53fa_c195_4625_bb1d_9310f983d9fb.slice/crio-1ad11ea710add76ec96405b779e414341247f9de6191a051068b1600259c2dda WatchSource:0}: Error finding container 1ad11ea710add76ec96405b779e414341247f9de6191a051068b1600259c2dda: Status 404 returned error can't find the container with id 1ad11ea710add76ec96405b779e414341247f9de6191a051068b1600259c2dda
Nov 04 03:23:18.082124 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:18.081105    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9tp7b" event={"ID":"179b6ae6-dd9d-4a69-8796-a1016dc2fef0","Type":"ContainerStarted","Data":"81efb9734979c75c7db27fded47b24ab4d18e2ba142bf668fa6920e464bb327a"}
Nov 04 03:23:18.082124 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:18.081175    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9tp7b" event={"ID":"179b6ae6-dd9d-4a69-8796-a1016dc2fef0","Type":"ContainerStarted","Data":"cd31a844f93e484ea98fa63b4d4453565105179b8c5be7c51dc1ee0e31ddeec4"}
Nov 04 03:23:18.089144 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:18.088635    5609 generic.go:356] "Generic (PLEG): container finished" podID="993ee084-01db-4f9e-8f56-fd3bd5a407a3" containerID="a1bba17626c1714d78856117a5f51dbc88cc165c9092d341dbbd0e8de2a9644d" exitCode=0
Nov 04 03:23:18.089144 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:18.088769    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c689lh" event={"ID":"993ee084-01db-4f9e-8f56-fd3bd5a407a3","Type":"ContainerDied","Data":"a1bba17626c1714d78856117a5f51dbc88cc165c9092d341dbbd0e8de2a9644d"}
Nov 04 03:23:18.089144 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:18.088827    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c689lh" event={"ID":"993ee084-01db-4f9e-8f56-fd3bd5a407a3","Type":"ContainerStarted","Data":"ad1506fbc911ddf45247260e273a7fab2b75d76f6c78ad0efcfb653e38c130c1"}
Nov 04 03:23:18.103856 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:18.103760    5609 generic.go:356] "Generic (PLEG): container finished" podID="7c0753e0-3eda-4c9b-a763-ca1c20982fed" containerID="98d23b155d08ba17afbbf1c30a0ba040cd662a6a11306a55b1bb2c0c5085d94a" exitCode=0
Nov 04 03:23:18.104093 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:18.104068    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvvl4h" event={"ID":"7c0753e0-3eda-4c9b-a763-ca1c20982fed","Type":"ContainerDied","Data":"98d23b155d08ba17afbbf1c30a0ba040cd662a6a11306a55b1bb2c0c5085d94a"}
Nov 04 03:23:18.113297 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:18.113268    5609 generic.go:356] "Generic (PLEG): container finished" podID="76d623cd-5534-4184-ae5e-de00ee4bc6c6" containerID="bbcfbe7bc95333a13e25df860225907bc5599c81811caaf85eb4eb432d15e6bc" exitCode=0
Nov 04 03:23:18.113717 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:18.113511    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64k9b5" event={"ID":"76d623cd-5534-4184-ae5e-de00ee4bc6c6","Type":"ContainerDied","Data":"bbcfbe7bc95333a13e25df860225907bc5599c81811caaf85eb4eb432d15e6bc"}
Nov 04 03:23:18.113717 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:18.113630    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64k9b5" event={"ID":"76d623cd-5534-4184-ae5e-de00ee4bc6c6","Type":"ContainerStarted","Data":"20cff31f6567026cd9404f99207e82cf5e57a873b1dc399557757928d61e2fa4"}
Nov 04 03:23:18.132901 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:18.132859    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pvbww" event={"ID":"184d53fa-c195-4625-bb1d-9310f983d9fb","Type":"ContainerStarted","Data":"1ad11ea710add76ec96405b779e414341247f9de6191a051068b1600259c2dda"}
Nov 04 03:23:18.435098 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:23:18.435058    5609 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod993ee084_01db_4f9e_8f56_fd3bd5a407a3.slice/crio-conmon-6da7be26b563a934399938d1954a31f3e81369dbf476b1046d4abb9567571d66.scope\": RecentStats: unable to find data in memory cache]"
Nov 04 03:23:18.590028 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:23:18.589926    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:23:19.156156 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:19.155203    5609 generic.go:356] "Generic (PLEG): container finished" podID="184d53fa-c195-4625-bb1d-9310f983d9fb" containerID="55afa9e4fec6e35a991e41cbaf180588dc95512dc76f4789b58fc5a6c3b0920e" exitCode=0
Nov 04 03:23:19.156156 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:19.155299    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pvbww" event={"ID":"184d53fa-c195-4625-bb1d-9310f983d9fb","Type":"ContainerDied","Data":"55afa9e4fec6e35a991e41cbaf180588dc95512dc76f4789b58fc5a6c3b0920e"}
Nov 04 03:23:19.167369 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:19.167334    5609 generic.go:356] "Generic (PLEG): container finished" podID="179b6ae6-dd9d-4a69-8796-a1016dc2fef0" containerID="81efb9734979c75c7db27fded47b24ab4d18e2ba142bf668fa6920e464bb327a" exitCode=0
Nov 04 03:23:19.167518 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:19.167460    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9tp7b" event={"ID":"179b6ae6-dd9d-4a69-8796-a1016dc2fef0","Type":"ContainerDied","Data":"81efb9734979c75c7db27fded47b24ab4d18e2ba142bf668fa6920e464bb327a"}
Nov 04 03:23:19.171986 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:19.171942    5609 generic.go:356] "Generic (PLEG): container finished" podID="993ee084-01db-4f9e-8f56-fd3bd5a407a3" containerID="6da7be26b563a934399938d1954a31f3e81369dbf476b1046d4abb9567571d66" exitCode=0
Nov 04 03:23:19.172092 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:19.172043    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c689lh" event={"ID":"993ee084-01db-4f9e-8f56-fd3bd5a407a3","Type":"ContainerDied","Data":"6da7be26b563a934399938d1954a31f3e81369dbf476b1046d4abb9567571d66"}
Nov 04 03:23:19.186724 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:19.186698    5609 generic.go:356] "Generic (PLEG): container finished" podID="76d623cd-5534-4184-ae5e-de00ee4bc6c6" containerID="2101bc939d7b617b93f09527c68651c6f51d7a66118b86dba0f2357fa89f25ef" exitCode=0
Nov 04 03:23:19.186833 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:19.186815    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64k9b5" event={"ID":"76d623cd-5534-4184-ae5e-de00ee4bc6c6","Type":"ContainerDied","Data":"2101bc939d7b617b93f09527c68651c6f51d7a66118b86dba0f2357fa89f25ef"}
Nov 04 03:23:20.214434 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:20.213313    5609 generic.go:356] "Generic (PLEG): container finished" podID="184d53fa-c195-4625-bb1d-9310f983d9fb" containerID="0c0b2c4db0d6645b39607622c0308f44d0da7b94dc54356d3f157a1200997f95" exitCode=0
Nov 04 03:23:20.214434 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:20.213455    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pvbww" event={"ID":"184d53fa-c195-4625-bb1d-9310f983d9fb","Type":"ContainerDied","Data":"0c0b2c4db0d6645b39607622c0308f44d0da7b94dc54356d3f157a1200997f95"}
Nov 04 03:23:20.219915 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:20.219884    5609 generic.go:356] "Generic (PLEG): container finished" podID="179b6ae6-dd9d-4a69-8796-a1016dc2fef0" containerID="d68f7831fb71a1562efeb6ee1b4f1eda2258d4a0a0d7a8286c303abb6f7b378d" exitCode=0
Nov 04 03:23:20.220032 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:20.219961    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9tp7b" event={"ID":"179b6ae6-dd9d-4a69-8796-a1016dc2fef0","Type":"ContainerDied","Data":"d68f7831fb71a1562efeb6ee1b4f1eda2258d4a0a0d7a8286c303abb6f7b378d"}
Nov 04 03:23:31.438697 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:31.438657    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c65smsf" event={"ID":"11891172-56b2-42f2-b44d-c60dbccf4db8","Type":"ContainerStarted","Data":"1c7781855311841ff16e60cda1594633354a79148dc48991ebd6b254e753d1fc"}
Nov 04 03:23:31.552600 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:31.552539    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c65smsf" podStartSLOduration=2.139084121 podStartE2EDuration="35.55252002s" totalImagesPullingTime="33.413435899s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:22:56 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:23:31.551882465 +0000 UTC m=+354.128535383" watchObservedRunningTime="2026-11-04 03:23:31.55252002 +0000 UTC m=+354.129172934"
Nov 04 03:23:32.458324 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:32.458287    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arwznk" event={"ID":"5056bae9-a44b-4fa1-83ce-61c5736b3490","Type":"ContainerStarted","Data":"476309c9ca666c269e65c0282ef794b128f5c2a27e6231feadf3db70a448b0b1"}
Nov 04 03:23:32.469651 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:32.469340    5609 generic.go:356] "Generic (PLEG): container finished" podID="11891172-56b2-42f2-b44d-c60dbccf4db8" containerID="1c7781855311841ff16e60cda1594633354a79148dc48991ebd6b254e753d1fc" exitCode=0
Nov 04 03:23:32.469651 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:32.469445    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c65smsf" event={"ID":"11891172-56b2-42f2-b44d-c60dbccf4db8","Type":"ContainerDied","Data":"1c7781855311841ff16e60cda1594633354a79148dc48991ebd6b254e753d1fc"}
Nov 04 03:23:32.688888 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:32.688817    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arwznk" podStartSLOduration=4.312909016 podStartE2EDuration="36.688795372s" totalImagesPullingTime="32.375886356s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:22:56 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:23:32.575350889 +0000 UTC m=+355.152003833" watchObservedRunningTime="2026-11-04 03:23:32.688795372 +0000 UTC m=+355.265448290"
Nov 04 03:23:33.488534 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:33.488498    5609 generic.go:356] "Generic (PLEG): container finished" podID="5056bae9-a44b-4fa1-83ce-61c5736b3490" containerID="476309c9ca666c269e65c0282ef794b128f5c2a27e6231feadf3db70a448b0b1" exitCode=0
Nov 04 03:23:33.489090 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:33.488608    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arwznk" event={"ID":"5056bae9-a44b-4fa1-83ce-61c5736b3490","Type":"ContainerDied","Data":"476309c9ca666c269e65c0282ef794b128f5c2a27e6231feadf3db70a448b0b1"}
Nov 04 03:23:33.504784 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:33.504735    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k7w5r" event={"ID":"beae230e-3fd0-41db-bcc8-8ec138d2c3db","Type":"ContainerStarted","Data":"0efaf31ef21625d87f2fe4bd9dfcca81fea5339a159942fd8953a93f1026b06f"}
Nov 04 03:23:33.721279 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:33.719940    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k7w5r" podStartSLOduration=3.7621565759999998 podStartE2EDuration="20.719921255s" totalImagesPullingTime="16.957764679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:23:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:23:33.7186461 +0000 UTC m=+356.295299015" watchObservedRunningTime="2026-11-04 03:23:33.719921255 +0000 UTC m=+356.296574170"
Nov 04 03:23:33.797594 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:33.797557    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c65smsf"
Nov 04 03:23:33.856770 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:33.855765    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/11891172-56b2-42f2-b44d-c60dbccf4db8-kube-api-access-7nscc\" (UniqueName: \"kubernetes.io/projected/11891172-56b2-42f2-b44d-c60dbccf4db8-kube-api-access-7nscc\") pod \"11891172-56b2-42f2-b44d-c60dbccf4db8\" (UID: \"11891172-56b2-42f2-b44d-c60dbccf4db8\") "
Nov 04 03:23:33.856770 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:33.855875    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/11891172-56b2-42f2-b44d-c60dbccf4db8-bundle\" (UniqueName: \"kubernetes.io/empty-dir/11891172-56b2-42f2-b44d-c60dbccf4db8-bundle\") pod \"11891172-56b2-42f2-b44d-c60dbccf4db8\" (UID: \"11891172-56b2-42f2-b44d-c60dbccf4db8\") "
Nov 04 03:23:33.856770 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:33.855907    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/11891172-56b2-42f2-b44d-c60dbccf4db8-util\" (UniqueName: \"kubernetes.io/empty-dir/11891172-56b2-42f2-b44d-c60dbccf4db8-util\") pod \"11891172-56b2-42f2-b44d-c60dbccf4db8\" (UID: \"11891172-56b2-42f2-b44d-c60dbccf4db8\") "
Nov 04 03:23:33.857712 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:33.857650    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11891172-56b2-42f2-b44d-c60dbccf4db8-bundle" pod "11891172-56b2-42f2-b44d-c60dbccf4db8" (UID: "11891172-56b2-42f2-b44d-c60dbccf4db8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:23:33.860066 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:33.860035    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11891172-56b2-42f2-b44d-c60dbccf4db8-kube-api-access-7nscc" pod "11891172-56b2-42f2-b44d-c60dbccf4db8" (UID: "11891172-56b2-42f2-b44d-c60dbccf4db8"). InnerVolumeSpecName "kube-api-access-7nscc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:23:33.865145 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:33.865024    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11891172-56b2-42f2-b44d-c60dbccf4db8-util" pod "11891172-56b2-42f2-b44d-c60dbccf4db8" (UID: "11891172-56b2-42f2-b44d-c60dbccf4db8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:23:33.957988 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:33.957937    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7nscc\" (UniqueName: \"kubernetes.io/projected/11891172-56b2-42f2-b44d-c60dbccf4db8-kube-api-access-7nscc\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:23:33.957988 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:33.957979    5609 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/11891172-56b2-42f2-b44d-c60dbccf4db8-bundle\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:23:33.957988 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:33.957994    5609 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/11891172-56b2-42f2-b44d-c60dbccf4db8-util\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:23:34.534798 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:34.534766    5609 generic.go:356] "Generic (PLEG): container finished" podID="07ece136-c4aa-4998-be0f-0099f6f1e590" containerID="ee7fe8e7ff8c2a4ec9e639dc9b50c790105d1d4ab8cefd78ff0540b7956d7904" exitCode=0
Nov 04 03:23:34.535211 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:34.534871    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m7vcx" event={"ID":"07ece136-c4aa-4998-be0f-0099f6f1e590","Type":"ContainerDied","Data":"ee7fe8e7ff8c2a4ec9e639dc9b50c790105d1d4ab8cefd78ff0540b7956d7904"}
Nov 04 03:23:34.544478 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:34.542072    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c65smsf" event={"ID":"11891172-56b2-42f2-b44d-c60dbccf4db8","Type":"ContainerDied","Data":"56a8721217efc05848659d0fc6c8438fdfbd8d5b0d87afd347ff5f4ef0498249"}
Nov 04 03:23:34.544478 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:34.542112    5609 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="56a8721217efc05848659d0fc6c8438fdfbd8d5b0d87afd347ff5f4ef0498249"
Nov 04 03:23:34.544478 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:34.542173    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c65smsf"
Nov 04 03:23:34.567199 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:34.566709    5609 generic.go:356] "Generic (PLEG): container finished" podID="beae230e-3fd0-41db-bcc8-8ec138d2c3db" containerID="0efaf31ef21625d87f2fe4bd9dfcca81fea5339a159942fd8953a93f1026b06f" exitCode=0
Nov 04 03:23:34.567199 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:34.567137    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k7w5r" event={"ID":"beae230e-3fd0-41db-bcc8-8ec138d2c3db","Type":"ContainerDied","Data":"0efaf31ef21625d87f2fe4bd9dfcca81fea5339a159942fd8953a93f1026b06f"}
Nov 04 03:23:34.822024 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:34.821308    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arwznk"
Nov 04 03:23:34.875320 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:34.875284    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5056bae9-a44b-4fa1-83ce-61c5736b3490-util\" (UniqueName: \"kubernetes.io/empty-dir/5056bae9-a44b-4fa1-83ce-61c5736b3490-util\") pod \"5056bae9-a44b-4fa1-83ce-61c5736b3490\" (UID: \"5056bae9-a44b-4fa1-83ce-61c5736b3490\") "
Nov 04 03:23:34.875527 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:34.875353    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5056bae9-a44b-4fa1-83ce-61c5736b3490-kube-api-access-frgc8\" (UniqueName: \"kubernetes.io/projected/5056bae9-a44b-4fa1-83ce-61c5736b3490-kube-api-access-frgc8\") pod \"5056bae9-a44b-4fa1-83ce-61c5736b3490\" (UID: \"5056bae9-a44b-4fa1-83ce-61c5736b3490\") "
Nov 04 03:23:34.875527 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:34.875444    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5056bae9-a44b-4fa1-83ce-61c5736b3490-bundle\" (UniqueName: \"kubernetes.io/empty-dir/5056bae9-a44b-4fa1-83ce-61c5736b3490-bundle\") pod \"5056bae9-a44b-4fa1-83ce-61c5736b3490\" (UID: \"5056bae9-a44b-4fa1-83ce-61c5736b3490\") "
Nov 04 03:23:34.876477 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:34.876445    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5056bae9-a44b-4fa1-83ce-61c5736b3490-bundle" pod "5056bae9-a44b-4fa1-83ce-61c5736b3490" (UID: "5056bae9-a44b-4fa1-83ce-61c5736b3490"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:23:34.884042 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:34.883982    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5056bae9-a44b-4fa1-83ce-61c5736b3490-util" pod "5056bae9-a44b-4fa1-83ce-61c5736b3490" (UID: "5056bae9-a44b-4fa1-83ce-61c5736b3490"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:23:34.891958 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:34.891916    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5056bae9-a44b-4fa1-83ce-61c5736b3490-kube-api-access-frgc8" pod "5056bae9-a44b-4fa1-83ce-61c5736b3490" (UID: "5056bae9-a44b-4fa1-83ce-61c5736b3490"). InnerVolumeSpecName "kube-api-access-frgc8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:23:34.976676 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:34.976552    5609 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5056bae9-a44b-4fa1-83ce-61c5736b3490-bundle\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:23:34.976676 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:34.976587    5609 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5056bae9-a44b-4fa1-83ce-61c5736b3490-util\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:23:34.976676 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:34.976601    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frgc8\" (UniqueName: \"kubernetes.io/projected/5056bae9-a44b-4fa1-83ce-61c5736b3490-kube-api-access-frgc8\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:23:35.576618 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:35.576424    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arwznk" event={"ID":"5056bae9-a44b-4fa1-83ce-61c5736b3490","Type":"ContainerDied","Data":"4a5206d2c279c509d6ef297dfb2b1613fc2e6becb30fdcbb7eab5879e4491af6"}
Nov 04 03:23:35.576618 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:35.576487    5609 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4a5206d2c279c509d6ef297dfb2b1613fc2e6becb30fdcbb7eab5879e4491af6"
Nov 04 03:23:35.576618 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:35.576437    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arwznk"
Nov 04 03:23:35.587312 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:35.587036    5609 generic.go:356] "Generic (PLEG): container finished" podID="7c0753e0-3eda-4c9b-a763-ca1c20982fed" containerID="340099545be0711957ff8830a179acc59e2781c51a9f620705994154ee1e7dcb" exitCode=0
Nov 04 03:23:35.587312 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:35.587122    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvvl4h" event={"ID":"7c0753e0-3eda-4c9b-a763-ca1c20982fed","Type":"ContainerDied","Data":"340099545be0711957ff8830a179acc59e2781c51a9f620705994154ee1e7dcb"}
Nov 04 03:23:35.591217 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:35.591192    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64k9b5" event={"ID":"76d623cd-5534-4184-ae5e-de00ee4bc6c6","Type":"ContainerStarted","Data":"66ce69e4aff7f8b8f562f5625aa5d7b465df41ef4bb3ad5fc32f4f1faa53c5c8"}
Nov 04 03:23:35.809805 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:35.809780    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k7w5r"
Nov 04 03:23:35.830549 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:35.828076    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m7vcx"
Nov 04 03:23:35.868471 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:35.868413    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64k9b5" podStartSLOduration=3.952880632 podStartE2EDuration="19.868386103s" totalImagesPullingTime="15.915505471s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:23:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:23:35.867379879 +0000 UTC m=+358.444032795" watchObservedRunningTime="2026-11-04 03:23:35.868386103 +0000 UTC m=+358.445039015"
Nov 04 03:23:35.885627 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:35.885354    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/beae230e-3fd0-41db-bcc8-8ec138d2c3db-bundle\" (UniqueName: \"kubernetes.io/empty-dir/beae230e-3fd0-41db-bcc8-8ec138d2c3db-bundle\") pod \"beae230e-3fd0-41db-bcc8-8ec138d2c3db\" (UID: \"beae230e-3fd0-41db-bcc8-8ec138d2c3db\") "
Nov 04 03:23:35.885627 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:35.885451    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/07ece136-c4aa-4998-be0f-0099f6f1e590-kube-api-access-6gvsv\" (UniqueName: \"kubernetes.io/projected/07ece136-c4aa-4998-be0f-0099f6f1e590-kube-api-access-6gvsv\") pod \"07ece136-c4aa-4998-be0f-0099f6f1e590\" (UID: \"07ece136-c4aa-4998-be0f-0099f6f1e590\") "
Nov 04 03:23:35.885627 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:35.885493    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/07ece136-c4aa-4998-be0f-0099f6f1e590-bundle\" (UniqueName: \"kubernetes.io/empty-dir/07ece136-c4aa-4998-be0f-0099f6f1e590-bundle\") pod \"07ece136-c4aa-4998-be0f-0099f6f1e590\" (UID: \"07ece136-c4aa-4998-be0f-0099f6f1e590\") "
Nov 04 03:23:35.885627 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:35.885521    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/beae230e-3fd0-41db-bcc8-8ec138d2c3db-kube-api-access-z29p4\" (UniqueName: \"kubernetes.io/projected/beae230e-3fd0-41db-bcc8-8ec138d2c3db-kube-api-access-z29p4\") pod \"beae230e-3fd0-41db-bcc8-8ec138d2c3db\" (UID: \"beae230e-3fd0-41db-bcc8-8ec138d2c3db\") "
Nov 04 03:23:35.886367 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:35.885947    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/07ece136-c4aa-4998-be0f-0099f6f1e590-util\" (UniqueName: \"kubernetes.io/empty-dir/07ece136-c4aa-4998-be0f-0099f6f1e590-util\") pod \"07ece136-c4aa-4998-be0f-0099f6f1e590\" (UID: \"07ece136-c4aa-4998-be0f-0099f6f1e590\") "
Nov 04 03:23:35.886367 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:35.886045    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/beae230e-3fd0-41db-bcc8-8ec138d2c3db-util\" (UniqueName: \"kubernetes.io/empty-dir/beae230e-3fd0-41db-bcc8-8ec138d2c3db-util\") pod \"beae230e-3fd0-41db-bcc8-8ec138d2c3db\" (UID: \"beae230e-3fd0-41db-bcc8-8ec138d2c3db\") "
Nov 04 03:23:35.886659 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:35.886618    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07ece136-c4aa-4998-be0f-0099f6f1e590-bundle" pod "07ece136-c4aa-4998-be0f-0099f6f1e590" (UID: "07ece136-c4aa-4998-be0f-0099f6f1e590"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:23:35.886812 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:35.886774    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/beae230e-3fd0-41db-bcc8-8ec138d2c3db-bundle" pod "beae230e-3fd0-41db-bcc8-8ec138d2c3db" (UID: "beae230e-3fd0-41db-bcc8-8ec138d2c3db"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:23:35.889097 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:35.889026    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/beae230e-3fd0-41db-bcc8-8ec138d2c3db-kube-api-access-z29p4" pod "beae230e-3fd0-41db-bcc8-8ec138d2c3db" (UID: "beae230e-3fd0-41db-bcc8-8ec138d2c3db"). InnerVolumeSpecName "kube-api-access-z29p4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:23:35.889328 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:35.889302    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07ece136-c4aa-4998-be0f-0099f6f1e590-kube-api-access-6gvsv" pod "07ece136-c4aa-4998-be0f-0099f6f1e590" (UID: "07ece136-c4aa-4998-be0f-0099f6f1e590"). InnerVolumeSpecName "kube-api-access-6gvsv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:23:35.891709 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:35.891676    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07ece136-c4aa-4998-be0f-0099f6f1e590-util" pod "07ece136-c4aa-4998-be0f-0099f6f1e590" (UID: "07ece136-c4aa-4998-be0f-0099f6f1e590"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:23:35.899608 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:35.899550    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/beae230e-3fd0-41db-bcc8-8ec138d2c3db-util" pod "beae230e-3fd0-41db-bcc8-8ec138d2c3db" (UID: "beae230e-3fd0-41db-bcc8-8ec138d2c3db"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:23:35.987262 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:35.987230    5609 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/beae230e-3fd0-41db-bcc8-8ec138d2c3db-bundle\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:23:35.987392 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:35.987268    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6gvsv\" (UniqueName: \"kubernetes.io/projected/07ece136-c4aa-4998-be0f-0099f6f1e590-kube-api-access-6gvsv\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:23:35.987392 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:35.987286    5609 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/07ece136-c4aa-4998-be0f-0099f6f1e590-bundle\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:23:35.987392 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:35.987300    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z29p4\" (UniqueName: \"kubernetes.io/projected/beae230e-3fd0-41db-bcc8-8ec138d2c3db-kube-api-access-z29p4\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:23:35.987392 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:35.987311    5609 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/07ece136-c4aa-4998-be0f-0099f6f1e590-util\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:23:35.987392 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:35.987324    5609 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/beae230e-3fd0-41db-bcc8-8ec138d2c3db-util\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:23:36.600276 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:36.600165    5609 generic.go:356] "Generic (PLEG): container finished" podID="76d623cd-5534-4184-ae5e-de00ee4bc6c6" containerID="66ce69e4aff7f8b8f562f5625aa5d7b465df41ef4bb3ad5fc32f4f1faa53c5c8" exitCode=0
Nov 04 03:23:36.600276 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:36.600210    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64k9b5" event={"ID":"76d623cd-5534-4184-ae5e-de00ee4bc6c6","Type":"ContainerDied","Data":"66ce69e4aff7f8b8f562f5625aa5d7b465df41ef4bb3ad5fc32f4f1faa53c5c8"}
Nov 04 03:23:36.603925 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:36.603684    5609 generic.go:356] "Generic (PLEG): container finished" podID="c1a5a9af-8221-4655-ba83-7da2acf5244e" containerID="fa3a28a73d9cb6fa359d76969a1b7e0aee23f7bdafcb6dfc7a27b8a96853e6f0" exitCode=0
Nov 04 03:23:36.603925 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:36.603762    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be712pstf" event={"ID":"c1a5a9af-8221-4655-ba83-7da2acf5244e","Type":"ContainerDied","Data":"fa3a28a73d9cb6fa359d76969a1b7e0aee23f7bdafcb6dfc7a27b8a96853e6f0"}
Nov 04 03:23:36.608437 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:36.606377    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m7vcx"
Nov 04 03:23:36.608437 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:36.606455    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m7vcx" event={"ID":"07ece136-c4aa-4998-be0f-0099f6f1e590","Type":"ContainerDied","Data":"86ff8af63a7c2fd7b245076cf5791a39296c48f095e41393cd0ac59ceee8fe67"}
Nov 04 03:23:36.608437 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:36.606496    5609 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="86ff8af63a7c2fd7b245076cf5791a39296c48f095e41393cd0ac59ceee8fe67"
Nov 04 03:23:36.611213 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:36.611192    5609 generic.go:356] "Generic (PLEG): container finished" podID="b860e253-749d-4c29-8aa5-0dae76b4d78b" containerID="85963fe7c70d3b661f2a436516780f4b981fbc8df5f2a3593e95250fadb1606d" exitCode=0
Nov 04 03:23:36.611328 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:36.611291    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctn42v" event={"ID":"b860e253-749d-4c29-8aa5-0dae76b4d78b","Type":"ContainerDied","Data":"85963fe7c70d3b661f2a436516780f4b981fbc8df5f2a3593e95250fadb1606d"}
Nov 04 03:23:36.614489 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:36.614339    5609 generic.go:356] "Generic (PLEG): container finished" podID="184d53fa-c195-4625-bb1d-9310f983d9fb" containerID="faa00d260d955c49e05618b7a56a3ec24acf88e11cfe68eed7c13e731e7f0ff0" exitCode=0
Nov 04 03:23:36.614489 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:36.614430    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pvbww" event={"ID":"184d53fa-c195-4625-bb1d-9310f983d9fb","Type":"ContainerDied","Data":"faa00d260d955c49e05618b7a56a3ec24acf88e11cfe68eed7c13e731e7f0ff0"}
Nov 04 03:23:36.617375 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:36.617351    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k7w5r" event={"ID":"beae230e-3fd0-41db-bcc8-8ec138d2c3db","Type":"ContainerDied","Data":"f57e90ee9a7d9e1e5a5e6bd08fe86efa922572d464cd0c6b4af6d9b10f89a90e"}
Nov 04 03:23:36.617375 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:36.617366    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k7w5r"
Nov 04 03:23:36.617585 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:36.617387    5609 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f57e90ee9a7d9e1e5a5e6bd08fe86efa922572d464cd0c6b4af6d9b10f89a90e"
Nov 04 03:23:36.626467 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:36.626436    5609 generic.go:356] "Generic (PLEG): container finished" podID="179b6ae6-dd9d-4a69-8796-a1016dc2fef0" containerID="6121c91db5f92e629eba0f9cc70f970344843111754e4907b4d143bc13967aa8" exitCode=0
Nov 04 03:23:36.626572 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:36.626552    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9tp7b" event={"ID":"179b6ae6-dd9d-4a69-8796-a1016dc2fef0","Type":"ContainerDied","Data":"6121c91db5f92e629eba0f9cc70f970344843111754e4907b4d143bc13967aa8"}
Nov 04 03:23:36.629110 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:36.629084    5609 generic.go:356] "Generic (PLEG): container finished" podID="993ee084-01db-4f9e-8f56-fd3bd5a407a3" containerID="2d27416c555389c27fe7368446ffa2428fa6bacd2eeead3af712d3d48ea72edd" exitCode=0
Nov 04 03:23:36.629199 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:36.629165    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c689lh" event={"ID":"993ee084-01db-4f9e-8f56-fd3bd5a407a3","Type":"ContainerDied","Data":"2d27416c555389c27fe7368446ffa2428fa6bacd2eeead3af712d3d48ea72edd"}
Nov 04 03:23:36.826009 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:36.825895    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvvl4h"
Nov 04 03:23:36.896091 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:36.896044    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7c0753e0-3eda-4c9b-a763-ca1c20982fed-util\" (UniqueName: \"kubernetes.io/empty-dir/7c0753e0-3eda-4c9b-a763-ca1c20982fed-util\") pod \"7c0753e0-3eda-4c9b-a763-ca1c20982fed\" (UID: \"7c0753e0-3eda-4c9b-a763-ca1c20982fed\") "
Nov 04 03:23:36.896296 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:36.896141    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7c0753e0-3eda-4c9b-a763-ca1c20982fed-bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c0753e0-3eda-4c9b-a763-ca1c20982fed-bundle\") pod \"7c0753e0-3eda-4c9b-a763-ca1c20982fed\" (UID: \"7c0753e0-3eda-4c9b-a763-ca1c20982fed\") "
Nov 04 03:23:36.896296 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:36.896278    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7c0753e0-3eda-4c9b-a763-ca1c20982fed-kube-api-access-485qt\" (UniqueName: \"kubernetes.io/projected/7c0753e0-3eda-4c9b-a763-ca1c20982fed-kube-api-access-485qt\") pod \"7c0753e0-3eda-4c9b-a763-ca1c20982fed\" (UID: \"7c0753e0-3eda-4c9b-a763-ca1c20982fed\") "
Nov 04 03:23:36.899985 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:36.899933    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c0753e0-3eda-4c9b-a763-ca1c20982fed-bundle" pod "7c0753e0-3eda-4c9b-a763-ca1c20982fed" (UID: "7c0753e0-3eda-4c9b-a763-ca1c20982fed"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:23:36.900097 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:36.900035    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c0753e0-3eda-4c9b-a763-ca1c20982fed-kube-api-access-485qt" pod "7c0753e0-3eda-4c9b-a763-ca1c20982fed" (UID: "7c0753e0-3eda-4c9b-a763-ca1c20982fed"). InnerVolumeSpecName "kube-api-access-485qt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:23:36.905253 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:36.905224    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c0753e0-3eda-4c9b-a763-ca1c20982fed-util" pod "7c0753e0-3eda-4c9b-a763-ca1c20982fed" (UID: "7c0753e0-3eda-4c9b-a763-ca1c20982fed"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:23:36.997414 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:36.997375    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-485qt\" (UniqueName: \"kubernetes.io/projected/7c0753e0-3eda-4c9b-a763-ca1c20982fed-kube-api-access-485qt\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:23:36.997414 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:36.997421    5609 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c0753e0-3eda-4c9b-a763-ca1c20982fed-util\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:23:36.997645 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:36.997436    5609 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c0753e0-3eda-4c9b-a763-ca1c20982fed-bundle\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:23:37.636592 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:37.636512    5609 generic.go:356] "Generic (PLEG): container finished" podID="c65ca1f2-8c4e-420d-8726-4d01a567a302" containerID="1d0c45bf029a1d5326467dc2a5ef14356ee797b96c5a7126d8d54e8fbd677aa1" exitCode=0
Nov 04 03:23:37.637038 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:37.636581    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448d75f" event={"ID":"c65ca1f2-8c4e-420d-8726-4d01a567a302","Type":"ContainerDied","Data":"1d0c45bf029a1d5326467dc2a5ef14356ee797b96c5a7126d8d54e8fbd677aa1"}
Nov 04 03:23:37.639111 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:37.639084    5609 generic.go:356] "Generic (PLEG): container finished" podID="a65be0cb-f19a-4303-9a56-f421a021f8eb" containerID="1faf2579ff8a2625357d45a4aeb6a3f71b58dfc11580b7709333fa86007f35a6" exitCode=0
Nov 04 03:23:37.639241 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:37.639132    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gdw75" event={"ID":"a65be0cb-f19a-4303-9a56-f421a021f8eb","Type":"ContainerDied","Data":"1faf2579ff8a2625357d45a4aeb6a3f71b58dfc11580b7709333fa86007f35a6"}
Nov 04 03:23:37.641220 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:37.641202    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvvl4h"
Nov 04 03:23:37.641321 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:37.641234    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvvl4h" event={"ID":"7c0753e0-3eda-4c9b-a763-ca1c20982fed","Type":"ContainerDied","Data":"4a31a2e7d30235b7f088d2ac5c06f93fb17757ffa47a7a012cb037110156c2d3"}
Nov 04 03:23:37.641321 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:37.641270    5609 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4a31a2e7d30235b7f088d2ac5c06f93fb17757ffa47a7a012cb037110156c2d3"
Nov 04 03:23:37.643623 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:37.643596    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cf7j9g" event={"ID":"4558182a-d695-4e09-8956-60c86a9c02f1","Type":"ContainerStarted","Data":"7d08e65596bfe4ba88893f8b4990b1858a51404489d5769acab8ffee548bd8f5"}
Nov 04 03:23:37.857528 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:37.857494    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9tp7b"
Nov 04 03:23:37.858866 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:37.858841    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c689lh"
Nov 04 03:23:37.865187 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:37.865161    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be712pstf"
Nov 04 03:23:37.894335 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:37.894277    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctn42v"
Nov 04 03:23:37.903816 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:37.903793    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pvbww"
Nov 04 03:23:37.908018 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:37.907998    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64k9b5"
Nov 04 03:23:37.946602 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:37.946551    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cf7j9g" podStartSLOduration=2.599868571 podStartE2EDuration="44.946535191s" totalImagesPullingTime="42.34666662s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:22:53 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:23:37.945681226 +0000 UTC m=+360.522334143" watchObservedRunningTime="2026-11-04 03:23:37.946535191 +0000 UTC m=+360.523188112"
Nov 04 03:23:38.008313 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.008281    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/993ee084-01db-4f9e-8f56-fd3bd5a407a3-bundle\" (UniqueName: \"kubernetes.io/empty-dir/993ee084-01db-4f9e-8f56-fd3bd5a407a3-bundle\") pod \"993ee084-01db-4f9e-8f56-fd3bd5a407a3\" (UID: \"993ee084-01db-4f9e-8f56-fd3bd5a407a3\") "
Nov 04 03:23:38.008517 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.008319    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1a5a9af-8221-4655-ba83-7da2acf5244e-bundle\" (UniqueName: \"kubernetes.io/empty-dir/c1a5a9af-8221-4655-ba83-7da2acf5244e-bundle\") pod \"c1a5a9af-8221-4655-ba83-7da2acf5244e\" (UID: \"c1a5a9af-8221-4655-ba83-7da2acf5244e\") "
Nov 04 03:23:38.008517 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.008347    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/179b6ae6-dd9d-4a69-8796-a1016dc2fef0-kube-api-access-fb6vl\" (UniqueName: \"kubernetes.io/projected/179b6ae6-dd9d-4a69-8796-a1016dc2fef0-kube-api-access-fb6vl\") pod \"179b6ae6-dd9d-4a69-8796-a1016dc2fef0\" (UID: \"179b6ae6-dd9d-4a69-8796-a1016dc2fef0\") "
Nov 04 03:23:38.008517 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.008364    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/179b6ae6-dd9d-4a69-8796-a1016dc2fef0-util\" (UniqueName: \"kubernetes.io/empty-dir/179b6ae6-dd9d-4a69-8796-a1016dc2fef0-util\") pod \"179b6ae6-dd9d-4a69-8796-a1016dc2fef0\" (UID: \"179b6ae6-dd9d-4a69-8796-a1016dc2fef0\") "
Nov 04 03:23:38.008517 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.008427    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/993ee084-01db-4f9e-8f56-fd3bd5a407a3-kube-api-access-spgcv\" (UniqueName: \"kubernetes.io/projected/993ee084-01db-4f9e-8f56-fd3bd5a407a3-kube-api-access-spgcv\") pod \"993ee084-01db-4f9e-8f56-fd3bd5a407a3\" (UID: \"993ee084-01db-4f9e-8f56-fd3bd5a407a3\") "
Nov 04 03:23:38.008749 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.008717    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/76d623cd-5534-4184-ae5e-de00ee4bc6c6-util\" (UniqueName: \"kubernetes.io/empty-dir/76d623cd-5534-4184-ae5e-de00ee4bc6c6-util\") pod \"76d623cd-5534-4184-ae5e-de00ee4bc6c6\" (UID: \"76d623cd-5534-4184-ae5e-de00ee4bc6c6\") "
Nov 04 03:23:38.008803 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.008766    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/76d623cd-5534-4184-ae5e-de00ee4bc6c6-bundle\" (UniqueName: \"kubernetes.io/empty-dir/76d623cd-5534-4184-ae5e-de00ee4bc6c6-bundle\") pod \"76d623cd-5534-4184-ae5e-de00ee4bc6c6\" (UID: \"76d623cd-5534-4184-ae5e-de00ee4bc6c6\") "
Nov 04 03:23:38.008857 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.008808    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1a5a9af-8221-4655-ba83-7da2acf5244e-util\" (UniqueName: \"kubernetes.io/empty-dir/c1a5a9af-8221-4655-ba83-7da2acf5244e-util\") pod \"c1a5a9af-8221-4655-ba83-7da2acf5244e\" (UID: \"c1a5a9af-8221-4655-ba83-7da2acf5244e\") "
Nov 04 03:23:38.008857 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.008840    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/993ee084-01db-4f9e-8f56-fd3bd5a407a3-util\" (UniqueName: \"kubernetes.io/empty-dir/993ee084-01db-4f9e-8f56-fd3bd5a407a3-util\") pod \"993ee084-01db-4f9e-8f56-fd3bd5a407a3\" (UID: \"993ee084-01db-4f9e-8f56-fd3bd5a407a3\") "
Nov 04 03:23:38.008952 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.008879    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c1a5a9af-8221-4655-ba83-7da2acf5244e-kube-api-access-hntdh\" (UniqueName: \"kubernetes.io/projected/c1a5a9af-8221-4655-ba83-7da2acf5244e-kube-api-access-hntdh\") pod \"c1a5a9af-8221-4655-ba83-7da2acf5244e\" (UID: \"c1a5a9af-8221-4655-ba83-7da2acf5244e\") "
Nov 04 03:23:38.008952 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.008906    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/76d623cd-5534-4184-ae5e-de00ee4bc6c6-kube-api-access-k9n2j\" (UniqueName: \"kubernetes.io/projected/76d623cd-5534-4184-ae5e-de00ee4bc6c6-kube-api-access-k9n2j\") pod \"76d623cd-5534-4184-ae5e-de00ee4bc6c6\" (UID: \"76d623cd-5534-4184-ae5e-de00ee4bc6c6\") "
Nov 04 03:23:38.009060 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.008952    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/184d53fa-c195-4625-bb1d-9310f983d9fb-kube-api-access-lmjfr\" (UniqueName: \"kubernetes.io/projected/184d53fa-c195-4625-bb1d-9310f983d9fb-kube-api-access-lmjfr\") pod \"184d53fa-c195-4625-bb1d-9310f983d9fb\" (UID: \"184d53fa-c195-4625-bb1d-9310f983d9fb\") "
Nov 04 03:23:38.009060 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.008992    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b860e253-749d-4c29-8aa5-0dae76b4d78b-bundle\" (UniqueName: \"kubernetes.io/empty-dir/b860e253-749d-4c29-8aa5-0dae76b4d78b-bundle\") pod \"b860e253-749d-4c29-8aa5-0dae76b4d78b\" (UID: \"b860e253-749d-4c29-8aa5-0dae76b4d78b\") "
Nov 04 03:23:38.009060 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.009016    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/179b6ae6-dd9d-4a69-8796-a1016dc2fef0-bundle\" (UniqueName: \"kubernetes.io/empty-dir/179b6ae6-dd9d-4a69-8796-a1016dc2fef0-bundle\") pod \"179b6ae6-dd9d-4a69-8796-a1016dc2fef0\" (UID: \"179b6ae6-dd9d-4a69-8796-a1016dc2fef0\") "
Nov 04 03:23:38.009197 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.009071    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b860e253-749d-4c29-8aa5-0dae76b4d78b-kube-api-access-pmtnx\" (UniqueName: \"kubernetes.io/projected/b860e253-749d-4c29-8aa5-0dae76b4d78b-kube-api-access-pmtnx\") pod \"b860e253-749d-4c29-8aa5-0dae76b4d78b\" (UID: \"b860e253-749d-4c29-8aa5-0dae76b4d78b\") "
Nov 04 03:23:38.009197 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.009107    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/184d53fa-c195-4625-bb1d-9310f983d9fb-util\" (UniqueName: \"kubernetes.io/empty-dir/184d53fa-c195-4625-bb1d-9310f983d9fb-util\") pod \"184d53fa-c195-4625-bb1d-9310f983d9fb\" (UID: \"184d53fa-c195-4625-bb1d-9310f983d9fb\") "
Nov 04 03:23:38.009197 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.009137    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/184d53fa-c195-4625-bb1d-9310f983d9fb-bundle\" (UniqueName: \"kubernetes.io/empty-dir/184d53fa-c195-4625-bb1d-9310f983d9fb-bundle\") pod \"184d53fa-c195-4625-bb1d-9310f983d9fb\" (UID: \"184d53fa-c195-4625-bb1d-9310f983d9fb\") "
Nov 04 03:23:38.009197 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.009188    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b860e253-749d-4c29-8aa5-0dae76b4d78b-util\" (UniqueName: \"kubernetes.io/empty-dir/b860e253-749d-4c29-8aa5-0dae76b4d78b-util\") pod \"b860e253-749d-4c29-8aa5-0dae76b4d78b\" (UID: \"b860e253-749d-4c29-8aa5-0dae76b4d78b\") "
Nov 04 03:23:38.009391 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.009358    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76d623cd-5534-4184-ae5e-de00ee4bc6c6-bundle" pod "76d623cd-5534-4184-ae5e-de00ee4bc6c6" (UID: "76d623cd-5534-4184-ae5e-de00ee4bc6c6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:23:38.009391 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.009375    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/993ee084-01db-4f9e-8f56-fd3bd5a407a3-bundle" pod "993ee084-01db-4f9e-8f56-fd3bd5a407a3" (UID: "993ee084-01db-4f9e-8f56-fd3bd5a407a3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:23:38.009523 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.009442    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1a5a9af-8221-4655-ba83-7da2acf5244e-bundle" pod "c1a5a9af-8221-4655-ba83-7da2acf5244e" (UID: "c1a5a9af-8221-4655-ba83-7da2acf5244e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:23:38.009830 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.009807    5609 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/993ee084-01db-4f9e-8f56-fd3bd5a407a3-bundle\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:23:38.009931 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.009835    5609 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c1a5a9af-8221-4655-ba83-7da2acf5244e-bundle\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:23:38.009931 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.009852    5609 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/76d623cd-5534-4184-ae5e-de00ee4bc6c6-bundle\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:23:38.010027 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.009968    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/184d53fa-c195-4625-bb1d-9310f983d9fb-bundle" pod "184d53fa-c195-4625-bb1d-9310f983d9fb" (UID: "184d53fa-c195-4625-bb1d-9310f983d9fb"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:23:38.010301 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.010279    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b860e253-749d-4c29-8aa5-0dae76b4d78b-bundle" pod "b860e253-749d-4c29-8aa5-0dae76b4d78b" (UID: "b860e253-749d-4c29-8aa5-0dae76b4d78b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:23:38.010943 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.010920    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/179b6ae6-dd9d-4a69-8796-a1016dc2fef0-bundle" pod "179b6ae6-dd9d-4a69-8796-a1016dc2fef0" (UID: "179b6ae6-dd9d-4a69-8796-a1016dc2fef0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:23:38.011982 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.011955    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/993ee084-01db-4f9e-8f56-fd3bd5a407a3-kube-api-access-spgcv" pod "993ee084-01db-4f9e-8f56-fd3bd5a407a3" (UID: "993ee084-01db-4f9e-8f56-fd3bd5a407a3"). InnerVolumeSpecName "kube-api-access-spgcv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:23:38.013537 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.013515    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b860e253-749d-4c29-8aa5-0dae76b4d78b-kube-api-access-pmtnx" pod "b860e253-749d-4c29-8aa5-0dae76b4d78b" (UID: "b860e253-749d-4c29-8aa5-0dae76b4d78b"). InnerVolumeSpecName "kube-api-access-pmtnx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:23:38.015519 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.015495    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/184d53fa-c195-4625-bb1d-9310f983d9fb-kube-api-access-lmjfr" pod "184d53fa-c195-4625-bb1d-9310f983d9fb" (UID: "184d53fa-c195-4625-bb1d-9310f983d9fb"). InnerVolumeSpecName "kube-api-access-lmjfr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:23:38.015794 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.015775    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1a5a9af-8221-4655-ba83-7da2acf5244e-kube-api-access-hntdh" pod "c1a5a9af-8221-4655-ba83-7da2acf5244e" (UID: "c1a5a9af-8221-4655-ba83-7da2acf5244e"). InnerVolumeSpecName "kube-api-access-hntdh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:23:38.017063 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.017038    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/179b6ae6-dd9d-4a69-8796-a1016dc2fef0-util" pod "179b6ae6-dd9d-4a69-8796-a1016dc2fef0" (UID: "179b6ae6-dd9d-4a69-8796-a1016dc2fef0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:23:38.017917 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.017883    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/184d53fa-c195-4625-bb1d-9310f983d9fb-util" pod "184d53fa-c195-4625-bb1d-9310f983d9fb" (UID: "184d53fa-c195-4625-bb1d-9310f983d9fb"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:23:38.019031 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.019007    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76d623cd-5534-4184-ae5e-de00ee4bc6c6-kube-api-access-k9n2j" pod "76d623cd-5534-4184-ae5e-de00ee4bc6c6" (UID: "76d623cd-5534-4184-ae5e-de00ee4bc6c6"). InnerVolumeSpecName "kube-api-access-k9n2j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:23:38.019502 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.019477    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1a5a9af-8221-4655-ba83-7da2acf5244e-util" pod "c1a5a9af-8221-4655-ba83-7da2acf5244e" (UID: "c1a5a9af-8221-4655-ba83-7da2acf5244e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:23:38.020331 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.020309    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b860e253-749d-4c29-8aa5-0dae76b4d78b-util" pod "b860e253-749d-4c29-8aa5-0dae76b4d78b" (UID: "b860e253-749d-4c29-8aa5-0dae76b4d78b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:23:38.020825 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.020806    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76d623cd-5534-4184-ae5e-de00ee4bc6c6-util" pod "76d623cd-5534-4184-ae5e-de00ee4bc6c6" (UID: "76d623cd-5534-4184-ae5e-de00ee4bc6c6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:23:38.021032 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.021016    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/179b6ae6-dd9d-4a69-8796-a1016dc2fef0-kube-api-access-fb6vl" pod "179b6ae6-dd9d-4a69-8796-a1016dc2fef0" (UID: "179b6ae6-dd9d-4a69-8796-a1016dc2fef0"). InnerVolumeSpecName "kube-api-access-fb6vl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:23:38.021176 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.021156    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/993ee084-01db-4f9e-8f56-fd3bd5a407a3-util" pod "993ee084-01db-4f9e-8f56-fd3bd5a407a3" (UID: "993ee084-01db-4f9e-8f56-fd3bd5a407a3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:23:38.110714 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.110688    5609 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c1a5a9af-8221-4655-ba83-7da2acf5244e-util\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:23:38.110714 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.110714    5609 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/993ee084-01db-4f9e-8f56-fd3bd5a407a3-util\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:23:38.110880 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.110723    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hntdh\" (UniqueName: \"kubernetes.io/projected/c1a5a9af-8221-4655-ba83-7da2acf5244e-kube-api-access-hntdh\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:23:38.110880 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.110732    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k9n2j\" (UniqueName: \"kubernetes.io/projected/76d623cd-5534-4184-ae5e-de00ee4bc6c6-kube-api-access-k9n2j\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:23:38.110880 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.110741    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lmjfr\" (UniqueName: \"kubernetes.io/projected/184d53fa-c195-4625-bb1d-9310f983d9fb-kube-api-access-lmjfr\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:23:38.110880 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.110751    5609 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b860e253-749d-4c29-8aa5-0dae76b4d78b-bundle\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:23:38.110880 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.110759    5609 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/179b6ae6-dd9d-4a69-8796-a1016dc2fef0-bundle\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:23:38.110880 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.110767    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pmtnx\" (UniqueName: \"kubernetes.io/projected/b860e253-749d-4c29-8aa5-0dae76b4d78b-kube-api-access-pmtnx\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:23:38.110880 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.110775    5609 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/184d53fa-c195-4625-bb1d-9310f983d9fb-util\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:23:38.110880 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.110782    5609 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/184d53fa-c195-4625-bb1d-9310f983d9fb-bundle\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:23:38.110880 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.110791    5609 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b860e253-749d-4c29-8aa5-0dae76b4d78b-util\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:23:38.110880 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.110798    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fb6vl\" (UniqueName: \"kubernetes.io/projected/179b6ae6-dd9d-4a69-8796-a1016dc2fef0-kube-api-access-fb6vl\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:23:38.110880 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.110805    5609 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/179b6ae6-dd9d-4a69-8796-a1016dc2fef0-util\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:23:38.110880 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.110814    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-spgcv\" (UniqueName: \"kubernetes.io/projected/993ee084-01db-4f9e-8f56-fd3bd5a407a3-kube-api-access-spgcv\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:23:38.110880 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.110822    5609 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/76d623cd-5534-4184-ae5e-de00ee4bc6c6-util\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:23:38.650872 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.650844    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctn42v"
Nov 04 03:23:38.650872 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.650847    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctn42v" event={"ID":"b860e253-749d-4c29-8aa5-0dae76b4d78b","Type":"ContainerDied","Data":"02e69e0ea114fb0c5c6aab99b6069702a1159e98ed4122f9ac42eea111e94413"}
Nov 04 03:23:38.651352 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.650904    5609 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="02e69e0ea114fb0c5c6aab99b6069702a1159e98ed4122f9ac42eea111e94413"
Nov 04 03:23:38.652864 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.652838    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pvbww"
Nov 04 03:23:38.653018 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.652844    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pvbww" event={"ID":"184d53fa-c195-4625-bb1d-9310f983d9fb","Type":"ContainerDied","Data":"1ad11ea710add76ec96405b779e414341247f9de6191a051068b1600259c2dda"}
Nov 04 03:23:38.653018 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.652952    5609 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1ad11ea710add76ec96405b779e414341247f9de6191a051068b1600259c2dda"
Nov 04 03:23:38.654913 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.654892    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9tp7b" event={"ID":"179b6ae6-dd9d-4a69-8796-a1016dc2fef0","Type":"ContainerDied","Data":"cd31a844f93e484ea98fa63b4d4453565105179b8c5be7c51dc1ee0e31ddeec4"}
Nov 04 03:23:38.655046 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.654911    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9tp7b"
Nov 04 03:23:38.655046 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.654926    5609 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cd31a844f93e484ea98fa63b4d4453565105179b8c5be7c51dc1ee0e31ddeec4"
Nov 04 03:23:38.656858 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.656836    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c689lh"
Nov 04 03:23:38.656976 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.656837    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c689lh" event={"ID":"993ee084-01db-4f9e-8f56-fd3bd5a407a3","Type":"ContainerDied","Data":"ad1506fbc911ddf45247260e273a7fab2b75d76f6c78ad0efcfb653e38c130c1"}
Nov 04 03:23:38.656976 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.656965    5609 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ad1506fbc911ddf45247260e273a7fab2b75d76f6c78ad0efcfb653e38c130c1"
Nov 04 03:23:38.659057 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.659037    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64k9b5"
Nov 04 03:23:38.659181 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.659053    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64k9b5" event={"ID":"76d623cd-5534-4184-ae5e-de00ee4bc6c6","Type":"ContainerDied","Data":"20cff31f6567026cd9404f99207e82cf5e57a873b1dc399557757928d61e2fa4"}
Nov 04 03:23:38.659181 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.659096    5609 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="20cff31f6567026cd9404f99207e82cf5e57a873b1dc399557757928d61e2fa4"
Nov 04 03:23:38.661515 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.661493    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be712pstf"
Nov 04 03:23:38.661515 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.661509    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be712pstf" event={"ID":"c1a5a9af-8221-4655-ba83-7da2acf5244e","Type":"ContainerDied","Data":"eec74d167c20425871d2f402afde0251b2de2be7920fb5b10f8063a80b1cb1d9"}
Nov 04 03:23:38.661755 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.661544    5609 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eec74d167c20425871d2f402afde0251b2de2be7920fb5b10f8063a80b1cb1d9"
Nov 04 03:23:38.670470 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.670447    5609 generic.go:356] "Generic (PLEG): container finished" podID="4558182a-d695-4e09-8956-60c86a9c02f1" containerID="7d08e65596bfe4ba88893f8b4990b1858a51404489d5769acab8ffee548bd8f5" exitCode=0
Nov 04 03:23:38.670798 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.670756    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cf7j9g" event={"ID":"4558182a-d695-4e09-8956-60c86a9c02f1","Type":"ContainerDied","Data":"7d08e65596bfe4ba88893f8b4990b1858a51404489d5769acab8ffee548bd8f5"}
Nov 04 03:23:38.833442 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.833416    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gdw75"
Nov 04 03:23:38.835949 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.835927    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448d75f"
Nov 04 03:23:38.917747 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.917663    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a65be0cb-f19a-4303-9a56-f421a021f8eb-kube-api-access-s8fkt\" (UniqueName: \"kubernetes.io/projected/a65be0cb-f19a-4303-9a56-f421a021f8eb-kube-api-access-s8fkt\") pod \"a65be0cb-f19a-4303-9a56-f421a021f8eb\" (UID: \"a65be0cb-f19a-4303-9a56-f421a021f8eb\") "
Nov 04 03:23:38.917929 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.917748    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c65ca1f2-8c4e-420d-8726-4d01a567a302-bundle\" (UniqueName: \"kubernetes.io/empty-dir/c65ca1f2-8c4e-420d-8726-4d01a567a302-bundle\") pod \"c65ca1f2-8c4e-420d-8726-4d01a567a302\" (UID: \"c65ca1f2-8c4e-420d-8726-4d01a567a302\") "
Nov 04 03:23:38.917929 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.917788    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a65be0cb-f19a-4303-9a56-f421a021f8eb-util\" (UniqueName: \"kubernetes.io/empty-dir/a65be0cb-f19a-4303-9a56-f421a021f8eb-util\") pod \"a65be0cb-f19a-4303-9a56-f421a021f8eb\" (UID: \"a65be0cb-f19a-4303-9a56-f421a021f8eb\") "
Nov 04 03:23:38.917929 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.917815    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a65be0cb-f19a-4303-9a56-f421a021f8eb-bundle\" (UniqueName: \"kubernetes.io/empty-dir/a65be0cb-f19a-4303-9a56-f421a021f8eb-bundle\") pod \"a65be0cb-f19a-4303-9a56-f421a021f8eb\" (UID: \"a65be0cb-f19a-4303-9a56-f421a021f8eb\") "
Nov 04 03:23:38.917929 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.917860    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c65ca1f2-8c4e-420d-8726-4d01a567a302-kube-api-access-j25x8\" (UniqueName: \"kubernetes.io/projected/c65ca1f2-8c4e-420d-8726-4d01a567a302-kube-api-access-j25x8\") pod \"c65ca1f2-8c4e-420d-8726-4d01a567a302\" (UID: \"c65ca1f2-8c4e-420d-8726-4d01a567a302\") "
Nov 04 03:23:38.917929 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.917890    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c65ca1f2-8c4e-420d-8726-4d01a567a302-util\" (UniqueName: \"kubernetes.io/empty-dir/c65ca1f2-8c4e-420d-8726-4d01a567a302-util\") pod \"c65ca1f2-8c4e-420d-8726-4d01a567a302\" (UID: \"c65ca1f2-8c4e-420d-8726-4d01a567a302\") "
Nov 04 03:23:38.918206 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.918174    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c65ca1f2-8c4e-420d-8726-4d01a567a302-bundle" pod "c65ca1f2-8c4e-420d-8726-4d01a567a302" (UID: "c65ca1f2-8c4e-420d-8726-4d01a567a302"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:23:38.918564 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.918540    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a65be0cb-f19a-4303-9a56-f421a021f8eb-bundle" pod "a65be0cb-f19a-4303-9a56-f421a021f8eb" (UID: "a65be0cb-f19a-4303-9a56-f421a021f8eb"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:23:38.919899 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.919871    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a65be0cb-f19a-4303-9a56-f421a021f8eb-kube-api-access-s8fkt" pod "a65be0cb-f19a-4303-9a56-f421a021f8eb" (UID: "a65be0cb-f19a-4303-9a56-f421a021f8eb"). InnerVolumeSpecName "kube-api-access-s8fkt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:23:38.919996 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.919951    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c65ca1f2-8c4e-420d-8726-4d01a567a302-kube-api-access-j25x8" pod "c65ca1f2-8c4e-420d-8726-4d01a567a302" (UID: "c65ca1f2-8c4e-420d-8726-4d01a567a302"). InnerVolumeSpecName "kube-api-access-j25x8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:23:38.924271 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.924249    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a65be0cb-f19a-4303-9a56-f421a021f8eb-util" pod "a65be0cb-f19a-4303-9a56-f421a021f8eb" (UID: "a65be0cb-f19a-4303-9a56-f421a021f8eb"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:23:38.924374 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:38.924323    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c65ca1f2-8c4e-420d-8726-4d01a567a302-util" pod "c65ca1f2-8c4e-420d-8726-4d01a567a302" (UID: "c65ca1f2-8c4e-420d-8726-4d01a567a302"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:23:39.018800 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:39.018765    5609 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c65ca1f2-8c4e-420d-8726-4d01a567a302-bundle\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:23:39.018800 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:39.018792    5609 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a65be0cb-f19a-4303-9a56-f421a021f8eb-util\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:23:39.018800 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:39.018800    5609 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a65be0cb-f19a-4303-9a56-f421a021f8eb-bundle\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:23:39.018997 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:39.018811    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j25x8\" (UniqueName: \"kubernetes.io/projected/c65ca1f2-8c4e-420d-8726-4d01a567a302-kube-api-access-j25x8\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:23:39.018997 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:39.018821    5609 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c65ca1f2-8c4e-420d-8726-4d01a567a302-util\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:23:39.018997 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:39.018830    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s8fkt\" (UniqueName: \"kubernetes.io/projected/a65be0cb-f19a-4303-9a56-f421a021f8eb-kube-api-access-s8fkt\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:23:39.682108 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:39.682058    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448d75f" event={"ID":"c65ca1f2-8c4e-420d-8726-4d01a567a302","Type":"ContainerDied","Data":"fbfdfbc540193717d15c8fc84398e58e697317932cf97a17daf175a12c1cd5ee"}
Nov 04 03:23:39.682108 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:39.682104    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448d75f"
Nov 04 03:23:39.682643 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:39.682150    5609 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fbfdfbc540193717d15c8fc84398e58e697317932cf97a17daf175a12c1cd5ee"
Nov 04 03:23:39.684378 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:39.684242    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gdw75" event={"ID":"a65be0cb-f19a-4303-9a56-f421a021f8eb","Type":"ContainerDied","Data":"4a67a2aae5b05b97c1e79be169e6ebf76a98aac82daae646c251f5fde09e88a6"}
Nov 04 03:23:39.684378 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:39.684284    5609 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4a67a2aae5b05b97c1e79be169e6ebf76a98aac82daae646c251f5fde09e88a6"
Nov 04 03:23:39.684378 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:39.684246    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gdw75"
Nov 04 03:23:39.807289 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:39.807261    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cf7j9g"
Nov 04 03:23:39.927425 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:39.927372    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4558182a-d695-4e09-8956-60c86a9c02f1-util\" (UniqueName: \"kubernetes.io/empty-dir/4558182a-d695-4e09-8956-60c86a9c02f1-util\") pod \"4558182a-d695-4e09-8956-60c86a9c02f1\" (UID: \"4558182a-d695-4e09-8956-60c86a9c02f1\") "
Nov 04 03:23:39.927581 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:39.927454    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4558182a-d695-4e09-8956-60c86a9c02f1-bundle\" (UniqueName: \"kubernetes.io/empty-dir/4558182a-d695-4e09-8956-60c86a9c02f1-bundle\") pod \"4558182a-d695-4e09-8956-60c86a9c02f1\" (UID: \"4558182a-d695-4e09-8956-60c86a9c02f1\") "
Nov 04 03:23:39.927581 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:39.927516    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4558182a-d695-4e09-8956-60c86a9c02f1-kube-api-access-9nxw6\" (UniqueName: \"kubernetes.io/projected/4558182a-d695-4e09-8956-60c86a9c02f1-kube-api-access-9nxw6\") pod \"4558182a-d695-4e09-8956-60c86a9c02f1\" (UID: \"4558182a-d695-4e09-8956-60c86a9c02f1\") "
Nov 04 03:23:39.928171 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:39.928139    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4558182a-d695-4e09-8956-60c86a9c02f1-bundle" pod "4558182a-d695-4e09-8956-60c86a9c02f1" (UID: "4558182a-d695-4e09-8956-60c86a9c02f1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:23:39.929613 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:39.929582    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4558182a-d695-4e09-8956-60c86a9c02f1-kube-api-access-9nxw6" pod "4558182a-d695-4e09-8956-60c86a9c02f1" (UID: "4558182a-d695-4e09-8956-60c86a9c02f1"). InnerVolumeSpecName "kube-api-access-9nxw6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:23:39.936028 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:39.936003    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4558182a-d695-4e09-8956-60c86a9c02f1-util" pod "4558182a-d695-4e09-8956-60c86a9c02f1" (UID: "4558182a-d695-4e09-8956-60c86a9c02f1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:23:40.029145 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.029109    5609 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4558182a-d695-4e09-8956-60c86a9c02f1-util\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:23:40.029145 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.029138    5609 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4558182a-d695-4e09-8956-60c86a9c02f1-bundle\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:23:40.029145 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.029150    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9nxw6\" (UniqueName: \"kubernetes.io/projected/4558182a-d695-4e09-8956-60c86a9c02f1-kube-api-access-9nxw6\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:23:40.691449 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.691414    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cf7j9g"
Nov 04 03:23:40.691862 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.691417    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cf7j9g" event={"ID":"4558182a-d695-4e09-8956-60c86a9c02f1","Type":"ContainerDied","Data":"f3d3f6ff2d1f656f425a04e1cf8f8d693491260d7f0641f40774143965ea267a"}
Nov 04 03:23:40.691862 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.691545    5609 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f3d3f6ff2d1f656f425a04e1cf8f8d693491260d7f0641f40774143965ea267a"
Nov 04 03:23:40.691941 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.691894    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-mtjbm"]
Nov 04 03:23:40.692175 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692162    5609 cpu_manager.go:414] "containerMap: removing container" podUID="a65be0cb-f19a-4303-9a56-f421a021f8eb" containerName="util"
Nov 04 03:23:40.692217 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692177    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a65be0cb-f19a-4303-9a56-f421a021f8eb" containerName="util"
Nov 04 03:23:40.692217 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692185    5609 cpu_manager.go:414] "containerMap: removing container" podUID="a65be0cb-f19a-4303-9a56-f421a021f8eb" containerName="pull"
Nov 04 03:23:40.692217 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692191    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a65be0cb-f19a-4303-9a56-f421a021f8eb" containerName="pull"
Nov 04 03:23:40.692217 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692199    5609 cpu_manager.go:414] "containerMap: removing container" podUID="993ee084-01db-4f9e-8f56-fd3bd5a407a3" containerName="util"
Nov 04 03:23:40.692217 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692203    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="993ee084-01db-4f9e-8f56-fd3bd5a407a3" containerName="util"
Nov 04 03:23:40.692217 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692211    5609 cpu_manager.go:414] "containerMap: removing container" podUID="11891172-56b2-42f2-b44d-c60dbccf4db8" containerName="pull"
Nov 04 03:23:40.692217 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692215    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="11891172-56b2-42f2-b44d-c60dbccf4db8" containerName="pull"
Nov 04 03:23:40.692441 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692222    5609 cpu_manager.go:414] "containerMap: removing container" podUID="7c0753e0-3eda-4c9b-a763-ca1c20982fed" containerName="pull"
Nov 04 03:23:40.692441 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692227    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7c0753e0-3eda-4c9b-a763-ca1c20982fed" containerName="pull"
Nov 04 03:23:40.692441 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692232    5609 cpu_manager.go:414] "containerMap: removing container" podUID="76d623cd-5534-4184-ae5e-de00ee4bc6c6" containerName="pull"
Nov 04 03:23:40.692441 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692236    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="76d623cd-5534-4184-ae5e-de00ee4bc6c6" containerName="pull"
Nov 04 03:23:40.692441 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692242    5609 cpu_manager.go:414] "containerMap: removing container" podUID="179b6ae6-dd9d-4a69-8796-a1016dc2fef0" containerName="pull"
Nov 04 03:23:40.692441 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692246    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="179b6ae6-dd9d-4a69-8796-a1016dc2fef0" containerName="pull"
Nov 04 03:23:40.692441 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692253    5609 cpu_manager.go:414] "containerMap: removing container" podUID="184d53fa-c195-4625-bb1d-9310f983d9fb" containerName="extract"
Nov 04 03:23:40.692441 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692257    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="184d53fa-c195-4625-bb1d-9310f983d9fb" containerName="extract"
Nov 04 03:23:40.692441 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692264    5609 cpu_manager.go:414] "containerMap: removing container" podUID="c65ca1f2-8c4e-420d-8726-4d01a567a302" containerName="pull"
Nov 04 03:23:40.692441 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692268    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c65ca1f2-8c4e-420d-8726-4d01a567a302" containerName="pull"
Nov 04 03:23:40.692441 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692275    5609 cpu_manager.go:414] "containerMap: removing container" podUID="c1a5a9af-8221-4655-ba83-7da2acf5244e" containerName="util"
Nov 04 03:23:40.692441 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692279    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c1a5a9af-8221-4655-ba83-7da2acf5244e" containerName="util"
Nov 04 03:23:40.692441 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692288    5609 cpu_manager.go:414] "containerMap: removing container" podUID="4558182a-d695-4e09-8956-60c86a9c02f1" containerName="extract"
Nov 04 03:23:40.692441 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692292    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4558182a-d695-4e09-8956-60c86a9c02f1" containerName="extract"
Nov 04 03:23:40.692441 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692299    5609 cpu_manager.go:414] "containerMap: removing container" podUID="b860e253-749d-4c29-8aa5-0dae76b4d78b" containerName="pull"
Nov 04 03:23:40.692441 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692303    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b860e253-749d-4c29-8aa5-0dae76b4d78b" containerName="pull"
Nov 04 03:23:40.692441 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692308    5609 cpu_manager.go:414] "containerMap: removing container" podUID="7c0753e0-3eda-4c9b-a763-ca1c20982fed" containerName="util"
Nov 04 03:23:40.692441 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692313    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7c0753e0-3eda-4c9b-a763-ca1c20982fed" containerName="util"
Nov 04 03:23:40.692441 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692318    5609 cpu_manager.go:414] "containerMap: removing container" podUID="b860e253-749d-4c29-8aa5-0dae76b4d78b" containerName="extract"
Nov 04 03:23:40.692441 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692322    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b860e253-749d-4c29-8aa5-0dae76b4d78b" containerName="extract"
Nov 04 03:23:40.692441 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692328    5609 cpu_manager.go:414] "containerMap: removing container" podUID="a65be0cb-f19a-4303-9a56-f421a021f8eb" containerName="extract"
Nov 04 03:23:40.692441 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692331    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a65be0cb-f19a-4303-9a56-f421a021f8eb" containerName="extract"
Nov 04 03:23:40.692441 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692346    5609 cpu_manager.go:414] "containerMap: removing container" podUID="5056bae9-a44b-4fa1-83ce-61c5736b3490" containerName="util"
Nov 04 03:23:40.692441 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692350    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5056bae9-a44b-4fa1-83ce-61c5736b3490" containerName="util"
Nov 04 03:23:40.692441 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692356    5609 cpu_manager.go:414] "containerMap: removing container" podUID="c1a5a9af-8221-4655-ba83-7da2acf5244e" containerName="pull"
Nov 04 03:23:40.692441 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692360    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c1a5a9af-8221-4655-ba83-7da2acf5244e" containerName="pull"
Nov 04 03:23:40.692441 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692366    5609 cpu_manager.go:414] "containerMap: removing container" podUID="5056bae9-a44b-4fa1-83ce-61c5736b3490" containerName="pull"
Nov 04 03:23:40.692441 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692371    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5056bae9-a44b-4fa1-83ce-61c5736b3490" containerName="pull"
Nov 04 03:23:40.692441 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692376    5609 cpu_manager.go:414] "containerMap: removing container" podUID="b860e253-749d-4c29-8aa5-0dae76b4d78b" containerName="util"
Nov 04 03:23:40.692441 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692380    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b860e253-749d-4c29-8aa5-0dae76b4d78b" containerName="util"
Nov 04 03:23:40.692441 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692387    5609 cpu_manager.go:414] "containerMap: removing container" podUID="07ece136-c4aa-4998-be0f-0099f6f1e590" containerName="pull"
Nov 04 03:23:40.692441 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692391    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="07ece136-c4aa-4998-be0f-0099f6f1e590" containerName="pull"
Nov 04 03:23:40.692441 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692411    5609 cpu_manager.go:414] "containerMap: removing container" podUID="993ee084-01db-4f9e-8f56-fd3bd5a407a3" containerName="pull"
Nov 04 03:23:40.692441 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692417    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="993ee084-01db-4f9e-8f56-fd3bd5a407a3" containerName="pull"
Nov 04 03:23:40.692441 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692427    5609 cpu_manager.go:414] "containerMap: removing container" podUID="7c0753e0-3eda-4c9b-a763-ca1c20982fed" containerName="extract"
Nov 04 03:23:40.692441 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692431    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7c0753e0-3eda-4c9b-a763-ca1c20982fed" containerName="extract"
Nov 04 03:23:40.692441 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692440    5609 cpu_manager.go:414] "containerMap: removing container" podUID="beae230e-3fd0-41db-bcc8-8ec138d2c3db" containerName="util"
Nov 04 03:23:40.692441 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692444    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="beae230e-3fd0-41db-bcc8-8ec138d2c3db" containerName="util"
Nov 04 03:23:40.692441 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692450    5609 cpu_manager.go:414] "containerMap: removing container" podUID="179b6ae6-dd9d-4a69-8796-a1016dc2fef0" containerName="util"
Nov 04 03:23:40.692441 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692455    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="179b6ae6-dd9d-4a69-8796-a1016dc2fef0" containerName="util"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692461    5609 cpu_manager.go:414] "containerMap: removing container" podUID="4558182a-d695-4e09-8956-60c86a9c02f1" containerName="util"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692465    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4558182a-d695-4e09-8956-60c86a9c02f1" containerName="util"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692471    5609 cpu_manager.go:414] "containerMap: removing container" podUID="4558182a-d695-4e09-8956-60c86a9c02f1" containerName="pull"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692475    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4558182a-d695-4e09-8956-60c86a9c02f1" containerName="pull"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692480    5609 cpu_manager.go:414] "containerMap: removing container" podUID="76d623cd-5534-4184-ae5e-de00ee4bc6c6" containerName="util"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692484    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="76d623cd-5534-4184-ae5e-de00ee4bc6c6" containerName="util"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692489    5609 cpu_manager.go:414] "containerMap: removing container" podUID="184d53fa-c195-4625-bb1d-9310f983d9fb" containerName="pull"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692492    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="184d53fa-c195-4625-bb1d-9310f983d9fb" containerName="pull"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692497    5609 cpu_manager.go:414] "containerMap: removing container" podUID="5056bae9-a44b-4fa1-83ce-61c5736b3490" containerName="extract"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692501    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5056bae9-a44b-4fa1-83ce-61c5736b3490" containerName="extract"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692508    5609 cpu_manager.go:414] "containerMap: removing container" podUID="c65ca1f2-8c4e-420d-8726-4d01a567a302" containerName="util"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692511    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c65ca1f2-8c4e-420d-8726-4d01a567a302" containerName="util"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692517    5609 cpu_manager.go:414] "containerMap: removing container" podUID="11891172-56b2-42f2-b44d-c60dbccf4db8" containerName="util"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692521    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="11891172-56b2-42f2-b44d-c60dbccf4db8" containerName="util"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692526    5609 cpu_manager.go:414] "containerMap: removing container" podUID="beae230e-3fd0-41db-bcc8-8ec138d2c3db" containerName="pull"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692529    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="beae230e-3fd0-41db-bcc8-8ec138d2c3db" containerName="pull"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692534    5609 cpu_manager.go:414] "containerMap: removing container" podUID="993ee084-01db-4f9e-8f56-fd3bd5a407a3" containerName="extract"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692538    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="993ee084-01db-4f9e-8f56-fd3bd5a407a3" containerName="extract"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692544    5609 cpu_manager.go:414] "containerMap: removing container" podUID="c65ca1f2-8c4e-420d-8726-4d01a567a302" containerName="extract"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692548    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c65ca1f2-8c4e-420d-8726-4d01a567a302" containerName="extract"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692554    5609 cpu_manager.go:414] "containerMap: removing container" podUID="07ece136-c4aa-4998-be0f-0099f6f1e590" containerName="util"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692558    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="07ece136-c4aa-4998-be0f-0099f6f1e590" containerName="util"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692564    5609 cpu_manager.go:414] "containerMap: removing container" podUID="11891172-56b2-42f2-b44d-c60dbccf4db8" containerName="extract"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692567    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="11891172-56b2-42f2-b44d-c60dbccf4db8" containerName="extract"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692573    5609 cpu_manager.go:414] "containerMap: removing container" podUID="beae230e-3fd0-41db-bcc8-8ec138d2c3db" containerName="extract"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692577    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="beae230e-3fd0-41db-bcc8-8ec138d2c3db" containerName="extract"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692582    5609 cpu_manager.go:414] "containerMap: removing container" podUID="76d623cd-5534-4184-ae5e-de00ee4bc6c6" containerName="extract"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692586    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="76d623cd-5534-4184-ae5e-de00ee4bc6c6" containerName="extract"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692592    5609 cpu_manager.go:414] "containerMap: removing container" podUID="179b6ae6-dd9d-4a69-8796-a1016dc2fef0" containerName="extract"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692596    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="179b6ae6-dd9d-4a69-8796-a1016dc2fef0" containerName="extract"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692603    5609 cpu_manager.go:414] "containerMap: removing container" podUID="184d53fa-c195-4625-bb1d-9310f983d9fb" containerName="util"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692607    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="184d53fa-c195-4625-bb1d-9310f983d9fb" containerName="util"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692612    5609 cpu_manager.go:414] "containerMap: removing container" podUID="07ece136-c4aa-4998-be0f-0099f6f1e590" containerName="extract"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692616    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="07ece136-c4aa-4998-be0f-0099f6f1e590" containerName="extract"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692622    5609 cpu_manager.go:414] "containerMap: removing container" podUID="c1a5a9af-8221-4655-ba83-7da2acf5244e" containerName="extract"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692626    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c1a5a9af-8221-4655-ba83-7da2acf5244e" containerName="extract"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692678    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="11891172-56b2-42f2-b44d-c60dbccf4db8" containerName="extract"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692685    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="7c0753e0-3eda-4c9b-a763-ca1c20982fed" containerName="extract"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692708    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="993ee084-01db-4f9e-8f56-fd3bd5a407a3" containerName="extract"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692715    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="c1a5a9af-8221-4655-ba83-7da2acf5244e" containerName="extract"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692721    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="c65ca1f2-8c4e-420d-8726-4d01a567a302" containerName="extract"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692726    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="5056bae9-a44b-4fa1-83ce-61c5736b3490" containerName="extract"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692733    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="beae230e-3fd0-41db-bcc8-8ec138d2c3db" containerName="extract"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692738    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="76d623cd-5534-4184-ae5e-de00ee4bc6c6" containerName="extract"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692744    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="179b6ae6-dd9d-4a69-8796-a1016dc2fef0" containerName="extract"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692749    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="07ece136-c4aa-4998-be0f-0099f6f1e590" containerName="extract"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692754    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="184d53fa-c195-4625-bb1d-9310f983d9fb" containerName="extract"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692760    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="4558182a-d695-4e09-8956-60c86a9c02f1" containerName="extract"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692765    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="a65be0cb-f19a-4303-9a56-f421a021f8eb" containerName="extract"
Nov 04 03:23:40.693736 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.692770    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="b860e253-749d-4c29-8aa5-0dae76b4d78b" containerName="extract"
Nov 04 03:23:40.695113 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.694908    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-mtjbm"
Nov 04 03:23:40.751714 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.751684    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-operator-dockercfg-gwwpc\""
Nov 04 03:23:40.798034 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.798000    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"kube-root-ca.crt\""
Nov 04 03:23:40.798034 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.798000    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"openshift-service-ca.crt\""
Nov 04 03:23:40.799671 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.799652    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-mtjbm"]
Nov 04 03:23:40.837168 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.837129    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkxt5\" (UniqueName: \"kubernetes.io/projected/9d43ce90-e08d-40f5-8908-8987ff57a198-kube-api-access-nkxt5\") pod \"jobset-operator-674cd59dc5-mtjbm\" (UID: \"9d43ce90-e08d-40f5-8908-8987ff57a198\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-mtjbm"
Nov 04 03:23:40.837341 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.837292    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9d43ce90-e08d-40f5-8908-8987ff57a198-tmp\") pod \"jobset-operator-674cd59dc5-mtjbm\" (UID: \"9d43ce90-e08d-40f5-8908-8987ff57a198\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-mtjbm"
Nov 04 03:23:40.937915 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.937870    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9d43ce90-e08d-40f5-8908-8987ff57a198-tmp\") pod \"jobset-operator-674cd59dc5-mtjbm\" (UID: \"9d43ce90-e08d-40f5-8908-8987ff57a198\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-mtjbm"
Nov 04 03:23:40.938097 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.937923    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkxt5\" (UniqueName: \"kubernetes.io/projected/9d43ce90-e08d-40f5-8908-8987ff57a198-kube-api-access-nkxt5\") pod \"jobset-operator-674cd59dc5-mtjbm\" (UID: \"9d43ce90-e08d-40f5-8908-8987ff57a198\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-mtjbm"
Nov 04 03:23:40.938275 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.938247    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9d43ce90-e08d-40f5-8908-8987ff57a198-tmp\") pod \"jobset-operator-674cd59dc5-mtjbm\" (UID: \"9d43ce90-e08d-40f5-8908-8987ff57a198\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-mtjbm"
Nov 04 03:23:40.996645 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:40.996556    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkxt5\" (UniqueName: \"kubernetes.io/projected/9d43ce90-e08d-40f5-8908-8987ff57a198-kube-api-access-nkxt5\") pod \"jobset-operator-674cd59dc5-mtjbm\" (UID: \"9d43ce90-e08d-40f5-8908-8987ff57a198\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-mtjbm"
Nov 04 03:23:41.015356 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:41.015324    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-mtjbm"
Nov 04 03:23:41.394214 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:41.394132    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-mtjbm"]
Nov 04 03:23:41.400178 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:23:41.400140    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9d43ce90_e08d_40f5_8908_8987ff57a198.slice/crio-c648f1b4737c6082b29431b998920553834b5d203019d188856c9c582c6fa07a WatchSource:0}: Error finding container c648f1b4737c6082b29431b998920553834b5d203019d188856c9c582c6fa07a: Status 404 returned error can't find the container with id c648f1b4737c6082b29431b998920553834b5d203019d188856c9c582c6fa07a
Nov 04 03:23:41.697309 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:41.697275    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-mtjbm" event={"ID":"9d43ce90-e08d-40f5-8908-8987ff57a198","Type":"ContainerStarted","Data":"c648f1b4737c6082b29431b998920553834b5d203019d188856c9c582c6fa07a"}
Nov 04 03:23:46.463902 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:46.463868    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-cj98l"]
Nov 04 03:23:46.466682 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:46.466666    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-cj98l"
Nov 04 03:23:46.532518 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:46.532486    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Nov 04 03:23:46.532740 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:46.532662    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Nov 04 03:23:46.532740 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:46.532681    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Nov 04 03:23:46.532849 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:46.532783    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Nov 04 03:23:46.532884 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:46.532845    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Nov 04 03:23:46.567844 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:46.567812    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-9kg9c\""
Nov 04 03:23:46.573627 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:46.573595    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-cj98l"]
Nov 04 03:23:46.589768 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:46.589740    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f4132643-6454-4455-9b22-810f013bd1f4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-cj98l\" (UID: \"f4132643-6454-4455-9b22-810f013bd1f4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-cj98l"
Nov 04 03:23:46.589891 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:46.589784    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f4132643-6454-4455-9b22-810f013bd1f4-metrics-cert\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-cj98l\" (UID: \"f4132643-6454-4455-9b22-810f013bd1f4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-cj98l"
Nov 04 03:23:46.589961 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:46.589899    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f4132643-6454-4455-9b22-810f013bd1f4-tmp\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-cj98l\" (UID: \"f4132643-6454-4455-9b22-810f013bd1f4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-cj98l"
Nov 04 03:23:46.589961 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:46.589938    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f4132643-6454-4455-9b22-810f013bd1f4-manifests\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-cj98l\" (UID: \"f4132643-6454-4455-9b22-810f013bd1f4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-cj98l"
Nov 04 03:23:46.590080 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:46.590038    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f4132643-6454-4455-9b22-810f013bd1f4-webhook-cert\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-cj98l\" (UID: \"f4132643-6454-4455-9b22-810f013bd1f4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-cj98l"
Nov 04 03:23:46.590114 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:46.590099    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfj55\" (UniqueName: \"kubernetes.io/projected/f4132643-6454-4455-9b22-810f013bd1f4-kube-api-access-mfj55\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-cj98l\" (UID: \"f4132643-6454-4455-9b22-810f013bd1f4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-cj98l"
Nov 04 03:23:46.691380 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:46.691348    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f4132643-6454-4455-9b22-810f013bd1f4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-cj98l\" (UID: \"f4132643-6454-4455-9b22-810f013bd1f4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-cj98l"
Nov 04 03:23:46.691550 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:46.691410    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f4132643-6454-4455-9b22-810f013bd1f4-metrics-cert\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-cj98l\" (UID: \"f4132643-6454-4455-9b22-810f013bd1f4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-cj98l"
Nov 04 03:23:46.691550 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:46.691461    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f4132643-6454-4455-9b22-810f013bd1f4-tmp\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-cj98l\" (UID: \"f4132643-6454-4455-9b22-810f013bd1f4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-cj98l"
Nov 04 03:23:46.691550 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:46.691497    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f4132643-6454-4455-9b22-810f013bd1f4-manifests\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-cj98l\" (UID: \"f4132643-6454-4455-9b22-810f013bd1f4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-cj98l"
Nov 04 03:23:46.691550 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:46.691529    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f4132643-6454-4455-9b22-810f013bd1f4-webhook-cert\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-cj98l\" (UID: \"f4132643-6454-4455-9b22-810f013bd1f4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-cj98l"
Nov 04 03:23:46.691770 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:46.691553    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfj55\" (UniqueName: \"kubernetes.io/projected/f4132643-6454-4455-9b22-810f013bd1f4-kube-api-access-mfj55\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-cj98l\" (UID: \"f4132643-6454-4455-9b22-810f013bd1f4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-cj98l"
Nov 04 03:23:46.691989 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:46.691954    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f4132643-6454-4455-9b22-810f013bd1f4-manifests\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-cj98l\" (UID: \"f4132643-6454-4455-9b22-810f013bd1f4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-cj98l"
Nov 04 03:23:46.693799 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:46.693770    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f4132643-6454-4455-9b22-810f013bd1f4-tmp\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-cj98l\" (UID: \"f4132643-6454-4455-9b22-810f013bd1f4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-cj98l"
Nov 04 03:23:46.694153 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:46.694134    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f4132643-6454-4455-9b22-810f013bd1f4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-cj98l\" (UID: \"f4132643-6454-4455-9b22-810f013bd1f4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-cj98l"
Nov 04 03:23:46.694217 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:46.694156    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f4132643-6454-4455-9b22-810f013bd1f4-metrics-cert\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-cj98l\" (UID: \"f4132643-6454-4455-9b22-810f013bd1f4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-cj98l"
Nov 04 03:23:46.694217 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:46.694164    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f4132643-6454-4455-9b22-810f013bd1f4-webhook-cert\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-cj98l\" (UID: \"f4132643-6454-4455-9b22-810f013bd1f4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-cj98l"
Nov 04 03:23:46.723503 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:46.723436    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-mtjbm" event={"ID":"9d43ce90-e08d-40f5-8908-8987ff57a198","Type":"ContainerStarted","Data":"8dd36da11dd9451f51a0ceda5601d703316ab39df4c17945a428d170bcbc5a7e"}
Nov 04 03:23:46.760313 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:46.760286    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfj55\" (UniqueName: \"kubernetes.io/projected/f4132643-6454-4455-9b22-810f013bd1f4-kube-api-access-mfj55\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-cj98l\" (UID: \"f4132643-6454-4455-9b22-810f013bd1f4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-cj98l"
Nov 04 03:23:46.780237 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:46.780212    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-cj98l"
Nov 04 03:23:47.270326 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:47.270244    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-mtjbm" podStartSLOduration=2.971080877 podStartE2EDuration="7.270224305s" totalImagesPullingTime="4.299143428s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:23:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:23:46.837624613 +0000 UTC m=+369.414277528" watchObservedRunningTime="2026-11-04 03:23:47.270224305 +0000 UTC m=+369.846877218"
Nov 04 03:23:47.271885 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:47.271857    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-cj98l"]
Nov 04 03:23:47.275004 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:23:47.274981    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4132643_6454_4455_9b22_810f013bd1f4.slice/crio-b060b4bf1eafaeca7d9d5f2d6b1bab847c3715a47864feac3c3095566f0b1839 WatchSource:0}: Error finding container b060b4bf1eafaeca7d9d5f2d6b1bab847c3715a47864feac3c3095566f0b1839: Status 404 returned error can't find the container with id b060b4bf1eafaeca7d9d5f2d6b1bab847c3715a47864feac3c3095566f0b1839
Nov 04 03:23:47.729906 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:47.729874    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-cj98l" event={"ID":"f4132643-6454-4455-9b22-810f013bd1f4","Type":"ContainerStarted","Data":"b060b4bf1eafaeca7d9d5f2d6b1bab847c3715a47864feac3c3095566f0b1839"}
Nov 04 03:23:48.564461 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:23:48.564419    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:23:51.361563 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:51.361522    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-q6ntf"]
Nov 04 03:23:51.366162 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:51.366139    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-q6ntf"
Nov 04 03:23:51.448085 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:51.448056    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-dpprr\""
Nov 04 03:23:51.465140 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:51.465104    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Nov 04 03:23:51.465304 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:51.465161    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Nov 04 03:23:51.467293 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:51.467266    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-q6ntf"]
Nov 04 03:23:51.533648 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:51.533611    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkjbw\" (UniqueName: \"kubernetes.io/projected/aac51478-14da-475d-aaf2-8a692291ed5e-kube-api-access-dkjbw\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-q6ntf\" (UID: \"aac51478-14da-475d-aaf2-8a692291ed5e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-q6ntf"
Nov 04 03:23:51.533809 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:51.533733    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aac51478-14da-475d-aaf2-8a692291ed5e-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-q6ntf\" (UID: \"aac51478-14da-475d-aaf2-8a692291ed5e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-q6ntf"
Nov 04 03:23:51.635091 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:51.635001    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aac51478-14da-475d-aaf2-8a692291ed5e-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-q6ntf\" (UID: \"aac51478-14da-475d-aaf2-8a692291ed5e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-q6ntf"
Nov 04 03:23:51.635091 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:51.635078    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkjbw\" (UniqueName: \"kubernetes.io/projected/aac51478-14da-475d-aaf2-8a692291ed5e-kube-api-access-dkjbw\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-q6ntf\" (UID: \"aac51478-14da-475d-aaf2-8a692291ed5e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-q6ntf"
Nov 04 03:23:51.635523 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:51.635487    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aac51478-14da-475d-aaf2-8a692291ed5e-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-q6ntf\" (UID: \"aac51478-14da-475d-aaf2-8a692291ed5e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-q6ntf"
Nov 04 03:23:51.697066 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:51.697028    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkjbw\" (UniqueName: \"kubernetes.io/projected/aac51478-14da-475d-aaf2-8a692291ed5e-kube-api-access-dkjbw\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-q6ntf\" (UID: \"aac51478-14da-475d-aaf2-8a692291ed5e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-q6ntf"
Nov 04 03:23:51.979657 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:51.979623    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-q6ntf"
Nov 04 03:23:52.456661 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:52.456623    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-q6ntf"]
Nov 04 03:23:52.462433 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:23:52.462388    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaac51478_14da_475d_aaf2_8a692291ed5e.slice/crio-1d0bfb784ab912793555952e81b1781ffbc4a06175e5ae2b3a8cee642f9631fc WatchSource:0}: Error finding container 1d0bfb784ab912793555952e81b1781ffbc4a06175e5ae2b3a8cee642f9631fc: Status 404 returned error can't find the container with id 1d0bfb784ab912793555952e81b1781ffbc4a06175e5ae2b3a8cee642f9631fc
Nov 04 03:23:52.758702 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:52.758663    5609 generic.go:356] "Generic (PLEG): container finished" podID="f4132643-6454-4455-9b22-810f013bd1f4" containerID="cb23b1426df0079bdcb3bad42c8936023d8ee754450fd9c39b07f7753258c296" exitCode=0
Nov 04 03:23:52.758884 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:52.758750    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-cj98l" event={"ID":"f4132643-6454-4455-9b22-810f013bd1f4","Type":"ContainerDied","Data":"cb23b1426df0079bdcb3bad42c8936023d8ee754450fd9c39b07f7753258c296"}
Nov 04 03:23:52.760309 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:52.760159    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-q6ntf" event={"ID":"aac51478-14da-475d-aaf2-8a692291ed5e","Type":"ContainerStarted","Data":"1d0bfb784ab912793555952e81b1781ffbc4a06175e5ae2b3a8cee642f9631fc"}
Nov 04 03:23:53.767786 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:53.767751    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-cj98l" event={"ID":"f4132643-6454-4455-9b22-810f013bd1f4","Type":"ContainerStarted","Data":"d698fa7b9d48802ac348c53d6a4b306eed3f8893300d8cac44924ecdfa6f1719"}
Nov 04 03:23:53.768196 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:53.767816    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-cj98l"
Nov 04 03:23:53.995591 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:53.995539    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-cj98l" podStartSLOduration=2.220518134 podStartE2EDuration="7.995522135s" totalImagesPullingTime="5.296758637s" totalInitContainerRuntime="478.245364ms" isStatefulPod=false podCreationTimestamp="2026-11-04 03:23:46 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:23:53.948619945 +0000 UTC m=+376.525272895" watchObservedRunningTime="2026-11-04 03:23:53.995522135 +0000 UTC m=+376.572175117"
Nov 04 03:23:55.780641 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:55.780603    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-q6ntf" event={"ID":"aac51478-14da-475d-aaf2-8a692291ed5e","Type":"ContainerStarted","Data":"04e5c9832f60752b95ee5a9731625795a3228130b4f9a0ee92fe6ca37a3b0c62"}
Nov 04 03:23:56.000547 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:23:56.000480    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-q6ntf" podStartSLOduration=2.049620699 podStartE2EDuration="5.000458756s" totalImagesPullingTime="2.950838057s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:23:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:23:55.951976927 +0000 UTC m=+378.528629839" watchObservedRunningTime="2026-11-04 03:23:56.000458756 +0000 UTC m=+378.577111688"
Nov 04 03:24:00.659023 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:00.658990    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-zttlw"]
Nov 04 03:24:00.663630 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:00.663610    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-zttlw"
Nov 04 03:24:00.713046 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:00.713015    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8l8w\" (UniqueName: \"kubernetes.io/projected/6923e755-81bb-447e-923a-ff5d745f8a84-kube-api-access-p8l8w\") pod \"dns-operator-controller-manager-8557cbdb84-zttlw\" (UID: \"6923e755-81bb-447e-923a-ff5d745f8a84\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-zttlw"
Nov 04 03:24:00.760287 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:00.760259    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Nov 04 03:24:00.760287 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:00.760259    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Nov 04 03:24:00.760539 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:00.760466    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Nov 04 03:24:00.760598 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:00.760552    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-bf7hx\""
Nov 04 03:24:00.762568 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:00.762545    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-zttlw"]
Nov 04 03:24:00.813895 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:00.813861    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8l8w\" (UniqueName: \"kubernetes.io/projected/6923e755-81bb-447e-923a-ff5d745f8a84-kube-api-access-p8l8w\") pod \"dns-operator-controller-manager-8557cbdb84-zttlw\" (UID: \"6923e755-81bb-447e-923a-ff5d745f8a84\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-zttlw"
Nov 04 03:24:00.872342 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:00.872308    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8l8w\" (UniqueName: \"kubernetes.io/projected/6923e755-81bb-447e-923a-ff5d745f8a84-kube-api-access-p8l8w\") pod \"dns-operator-controller-manager-8557cbdb84-zttlw\" (UID: \"6923e755-81bb-447e-923a-ff5d745f8a84\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-zttlw"
Nov 04 03:24:00.975696 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:00.975666    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-zttlw"
Nov 04 03:24:01.348925 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:01.348839    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-zttlw"]
Nov 04 03:24:01.354654 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:24:01.354614    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6923e755_81bb_447e_923a_ff5d745f8a84.slice/crio-78b0e16e98b352228d5d60e601ce48317063c9060224f8b90f445e84b8fb18ec WatchSource:0}: Error finding container 78b0e16e98b352228d5d60e601ce48317063c9060224f8b90f445e84b8fb18ec: Status 404 returned error can't find the container with id 78b0e16e98b352228d5d60e601ce48317063c9060224f8b90f445e84b8fb18ec
Nov 04 03:24:01.813283 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:01.813241    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-zttlw" event={"ID":"6923e755-81bb-447e-923a-ff5d745f8a84","Type":"ContainerStarted","Data":"78b0e16e98b352228d5d60e601ce48317063c9060224f8b90f445e84b8fb18ec"}
Nov 04 03:24:04.776774 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:04.776741    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-cj98l"
Nov 04 03:24:08.508479 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:08.508443    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-4hmfk"]
Nov 04 03:24:08.515447 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:08.515394    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-4hmfk"
Nov 04 03:24:08.555434 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:08.555393    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-4hmfk"]
Nov 04 03:24:08.580615 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:08.580583    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgsg5\" (UniqueName: \"kubernetes.io/projected/923229c3-ab54-46ca-ac3a-fd9e951116c7-kube-api-access-vgsg5\") pod \"kuadrant-operator-controller-manager-678d87766f-4hmfk\" (UID: \"923229c3-ab54-46ca-ac3a-fd9e951116c7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-4hmfk"
Nov 04 03:24:08.580793 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:08.580653    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/923229c3-ab54-46ca-ac3a-fd9e951116c7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-4hmfk\" (UID: \"923229c3-ab54-46ca-ac3a-fd9e951116c7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-4hmfk"
Nov 04 03:24:08.600866 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:08.600839    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-6thkp\""
Nov 04 03:24:08.681594 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:08.681560    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgsg5\" (UniqueName: \"kubernetes.io/projected/923229c3-ab54-46ca-ac3a-fd9e951116c7-kube-api-access-vgsg5\") pod \"kuadrant-operator-controller-manager-678d87766f-4hmfk\" (UID: \"923229c3-ab54-46ca-ac3a-fd9e951116c7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-4hmfk"
Nov 04 03:24:08.681743 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:08.681614    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/923229c3-ab54-46ca-ac3a-fd9e951116c7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-4hmfk\" (UID: \"923229c3-ab54-46ca-ac3a-fd9e951116c7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-4hmfk"
Nov 04 03:24:08.681972 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:08.681955    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/923229c3-ab54-46ca-ac3a-fd9e951116c7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-4hmfk\" (UID: \"923229c3-ab54-46ca-ac3a-fd9e951116c7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-4hmfk"
Nov 04 03:24:08.742061 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:08.742038    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgsg5\" (UniqueName: \"kubernetes.io/projected/923229c3-ab54-46ca-ac3a-fd9e951116c7-kube-api-access-vgsg5\") pod \"kuadrant-operator-controller-manager-678d87766f-4hmfk\" (UID: \"923229c3-ab54-46ca-ac3a-fd9e951116c7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-4hmfk"
Nov 04 03:24:08.829280 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:08.829208    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-4hmfk"
Nov 04 03:24:09.304974 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:09.304941    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-4hmfk"]
Nov 04 03:24:09.311484 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:24:09.311456    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod923229c3_ab54_46ca_ac3a_fd9e951116c7.slice/crio-19607a50fc41219fcbcbf161eb7fee2607b8e99f131832fee3175cbe94148fb1 WatchSource:0}: Error finding container 19607a50fc41219fcbcbf161eb7fee2607b8e99f131832fee3175cbe94148fb1: Status 404 returned error can't find the container with id 19607a50fc41219fcbcbf161eb7fee2607b8e99f131832fee3175cbe94148fb1
Nov 04 03:24:09.855256 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:09.855221    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-4hmfk" event={"ID":"923229c3-ab54-46ca-ac3a-fd9e951116c7","Type":"ContainerStarted","Data":"19607a50fc41219fcbcbf161eb7fee2607b8e99f131832fee3175cbe94148fb1"}
Nov 04 03:24:12.163717 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:12.163675    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-hshfp"]
Nov 04 03:24:12.167284 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:12.167214    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-7b7bb4bc77-hshfp"
Nov 04 03:24:12.211897 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:12.211863    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxw8d\" (UniqueName: \"kubernetes.io/projected/894e9896-b132-40b9-9fb2-8a5c7f60703c-kube-api-access-gxw8d\") pod \"authorino-operator-7b7bb4bc77-hshfp\" (UID: \"894e9896-b132-40b9-9fb2-8a5c7f60703c\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-hshfp"
Nov 04 03:24:12.267351 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:12.267314    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-hshfp"]
Nov 04 03:24:12.267520 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:12.267485    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-hjpll\""
Nov 04 03:24:12.312645 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:12.312575    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxw8d\" (UniqueName: \"kubernetes.io/projected/894e9896-b132-40b9-9fb2-8a5c7f60703c-kube-api-access-gxw8d\") pod \"authorino-operator-7b7bb4bc77-hshfp\" (UID: \"894e9896-b132-40b9-9fb2-8a5c7f60703c\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-hshfp"
Nov 04 03:24:12.376159 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:12.376133    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxw8d\" (UniqueName: \"kubernetes.io/projected/894e9896-b132-40b9-9fb2-8a5c7f60703c-kube-api-access-gxw8d\") pod \"authorino-operator-7b7bb4bc77-hshfp\" (UID: \"894e9896-b132-40b9-9fb2-8a5c7f60703c\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-hshfp"
Nov 04 03:24:12.484705 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:12.484659    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-7b7bb4bc77-hshfp"
Nov 04 03:24:12.873653 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:12.873568    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-hshfp"]
Nov 04 03:24:12.908166 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:24:12.908128    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod894e9896_b132_40b9_9fb2_8a5c7f60703c.slice/crio-a939445f16d8ffdb6f82a122ee4847d5758d21041315b602fe6e3287b016d24b WatchSource:0}: Error finding container a939445f16d8ffdb6f82a122ee4847d5758d21041315b602fe6e3287b016d24b: Status 404 returned error can't find the container with id a939445f16d8ffdb6f82a122ee4847d5758d21041315b602fe6e3287b016d24b
Nov 04 03:24:13.880472 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:13.880432    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-7b7bb4bc77-hshfp" event={"ID":"894e9896-b132-40b9-9fb2-8a5c7f60703c","Type":"ContainerStarted","Data":"a939445f16d8ffdb6f82a122ee4847d5758d21041315b602fe6e3287b016d24b"}
Nov 04 03:24:15.895829 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:15.895733    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-4hmfk" event={"ID":"923229c3-ab54-46ca-ac3a-fd9e951116c7","Type":"ContainerStarted","Data":"0327d9b7aaa7f1d31f711206a8342d0a4cf0fed74552c886e74505abb5258b82"}
Nov 04 03:24:15.896264 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:15.895877    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-4hmfk"
Nov 04 03:24:16.105217 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:16.105164    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-4hmfk" podStartSLOduration=1.8452543970000002 podStartE2EDuration="8.105147716s" totalImagesPullingTime="6.259893319s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:24:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:24:16.058100172 +0000 UTC m=+398.634753086" watchObservedRunningTime="2026-11-04 03:24:16.105147716 +0000 UTC m=+398.681800631"
Nov 04 03:24:16.903267 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:16.903193    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-7b7bb4bc77-hshfp" event={"ID":"894e9896-b132-40b9-9fb2-8a5c7f60703c","Type":"ContainerStarted","Data":"d72193513b3091e91ade9ef05fd34d0001d7a6deb3f54b008c875eaff1b9193a"}
Nov 04 03:24:16.903789 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:16.903308    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-7b7bb4bc77-hshfp"
Nov 04 03:24:17.018313 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:17.018255    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-7b7bb4bc77-hshfp" podStartSLOduration=1.862782407 podStartE2EDuration="5.018237992s" totalImagesPullingTime="3.155455585s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:24:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:24:17.016447109 +0000 UTC m=+399.593100025" watchObservedRunningTime="2026-11-04 03:24:17.018237992 +0000 UTC m=+399.594890905"
Nov 04 03:24:17.909439 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:17.909390    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-zttlw" event={"ID":"6923e755-81bb-447e-923a-ff5d745f8a84","Type":"ContainerStarted","Data":"45a247dee0b3e0cd2049fd366b45541526b2bf1e60edfdae948fd3a6cde2100f"}
Nov 04 03:24:17.909882 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:17.909479    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-zttlw"
Nov 04 03:24:18.022069 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:18.022016    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-zttlw" podStartSLOduration=2.220091457 podStartE2EDuration="18.022001744s" totalImagesPullingTime="15.801910287s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:24:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:24:18.020212548 +0000 UTC m=+400.596865459" watchObservedRunningTime="2026-11-04 03:24:18.022001744 +0000 UTC m=+400.598654656"
Nov 04 03:24:18.563578 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:24:18.563541    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:24:26.905810 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:26.905773    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-4hmfk"
Nov 04 03:24:27.912574 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:27.912543    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-7b7bb4bc77-hshfp"
Nov 04 03:24:28.917820 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:28.917788    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-zttlw"
Nov 04 03:24:35.757990 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:35.757953    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c"]
Nov 04 03:24:35.767578 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:35.767553    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c"
Nov 04 03:24:35.834777 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:35.834748    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c"]
Nov 04 03:24:35.867725 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:35.867698    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-bjlr4\""
Nov 04 03:24:35.919354 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:35.919322    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/416ff65b-51ec-42e2-a4da-071eb613eac6-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c\" (UID: \"416ff65b-51ec-42e2-a4da-071eb613eac6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c"
Nov 04 03:24:35.919537 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:35.919361    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/416ff65b-51ec-42e2-a4da-071eb613eac6-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c\" (UID: \"416ff65b-51ec-42e2-a4da-071eb613eac6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c"
Nov 04 03:24:35.919537 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:35.919457    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/416ff65b-51ec-42e2-a4da-071eb613eac6-istio-data\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c\" (UID: \"416ff65b-51ec-42e2-a4da-071eb613eac6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c"
Nov 04 03:24:35.919537 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:35.919510    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/416ff65b-51ec-42e2-a4da-071eb613eac6-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c\" (UID: \"416ff65b-51ec-42e2-a4da-071eb613eac6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c"
Nov 04 03:24:35.919644 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:35.919554    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/416ff65b-51ec-42e2-a4da-071eb613eac6-istio-token\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c\" (UID: \"416ff65b-51ec-42e2-a4da-071eb613eac6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c"
Nov 04 03:24:35.919644 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:35.919582    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sffk8\" (UniqueName: \"kubernetes.io/projected/416ff65b-51ec-42e2-a4da-071eb613eac6-kube-api-access-sffk8\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c\" (UID: \"416ff65b-51ec-42e2-a4da-071eb613eac6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c"
Nov 04 03:24:35.919704 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:35.919663    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/416ff65b-51ec-42e2-a4da-071eb613eac6-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c\" (UID: \"416ff65b-51ec-42e2-a4da-071eb613eac6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c"
Nov 04 03:24:35.919704 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:35.919691    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/416ff65b-51ec-42e2-a4da-071eb613eac6-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c\" (UID: \"416ff65b-51ec-42e2-a4da-071eb613eac6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c"
Nov 04 03:24:35.919763 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:35.919706    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/416ff65b-51ec-42e2-a4da-071eb613eac6-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c\" (UID: \"416ff65b-51ec-42e2-a4da-071eb613eac6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c"
Nov 04 03:24:36.021199 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:36.021108    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/416ff65b-51ec-42e2-a4da-071eb613eac6-istio-data\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c\" (UID: \"416ff65b-51ec-42e2-a4da-071eb613eac6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c"
Nov 04 03:24:36.021199 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:36.021148    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/416ff65b-51ec-42e2-a4da-071eb613eac6-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c\" (UID: \"416ff65b-51ec-42e2-a4da-071eb613eac6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c"
Nov 04 03:24:36.021199 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:36.021168    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/416ff65b-51ec-42e2-a4da-071eb613eac6-istio-token\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c\" (UID: \"416ff65b-51ec-42e2-a4da-071eb613eac6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c"
Nov 04 03:24:36.021199 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:36.021190    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sffk8\" (UniqueName: \"kubernetes.io/projected/416ff65b-51ec-42e2-a4da-071eb613eac6-kube-api-access-sffk8\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c\" (UID: \"416ff65b-51ec-42e2-a4da-071eb613eac6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c"
Nov 04 03:24:36.021594 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:36.021241    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/416ff65b-51ec-42e2-a4da-071eb613eac6-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c\" (UID: \"416ff65b-51ec-42e2-a4da-071eb613eac6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c"
Nov 04 03:24:36.021594 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:36.021267    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/416ff65b-51ec-42e2-a4da-071eb613eac6-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c\" (UID: \"416ff65b-51ec-42e2-a4da-071eb613eac6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c"
Nov 04 03:24:36.021594 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:36.021285    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/416ff65b-51ec-42e2-a4da-071eb613eac6-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c\" (UID: \"416ff65b-51ec-42e2-a4da-071eb613eac6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c"
Nov 04 03:24:36.021594 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:36.021331    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/416ff65b-51ec-42e2-a4da-071eb613eac6-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c\" (UID: \"416ff65b-51ec-42e2-a4da-071eb613eac6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c"
Nov 04 03:24:36.021594 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:36.021360    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/416ff65b-51ec-42e2-a4da-071eb613eac6-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c\" (UID: \"416ff65b-51ec-42e2-a4da-071eb613eac6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c"
Nov 04 03:24:36.021804 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:36.021649    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/416ff65b-51ec-42e2-a4da-071eb613eac6-istio-data\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c\" (UID: \"416ff65b-51ec-42e2-a4da-071eb613eac6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c"
Nov 04 03:24:36.021804 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:36.021731    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/416ff65b-51ec-42e2-a4da-071eb613eac6-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c\" (UID: \"416ff65b-51ec-42e2-a4da-071eb613eac6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c"
Nov 04 03:24:36.021804 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:36.021770    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/416ff65b-51ec-42e2-a4da-071eb613eac6-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c\" (UID: \"416ff65b-51ec-42e2-a4da-071eb613eac6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c"
Nov 04 03:24:36.022048 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:36.022027    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/416ff65b-51ec-42e2-a4da-071eb613eac6-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c\" (UID: \"416ff65b-51ec-42e2-a4da-071eb613eac6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c"
Nov 04 03:24:36.022148 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:36.022132    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/416ff65b-51ec-42e2-a4da-071eb613eac6-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c\" (UID: \"416ff65b-51ec-42e2-a4da-071eb613eac6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c"
Nov 04 03:24:36.023544 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:36.023526    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/416ff65b-51ec-42e2-a4da-071eb613eac6-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c\" (UID: \"416ff65b-51ec-42e2-a4da-071eb613eac6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c"
Nov 04 03:24:36.023837 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:36.023821    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/416ff65b-51ec-42e2-a4da-071eb613eac6-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c\" (UID: \"416ff65b-51ec-42e2-a4da-071eb613eac6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c"
Nov 04 03:24:36.079540 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:36.079502    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/416ff65b-51ec-42e2-a4da-071eb613eac6-istio-token\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c\" (UID: \"416ff65b-51ec-42e2-a4da-071eb613eac6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c"
Nov 04 03:24:36.079714 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:36.079657    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sffk8\" (UniqueName: \"kubernetes.io/projected/416ff65b-51ec-42e2-a4da-071eb613eac6-kube-api-access-sffk8\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c\" (UID: \"416ff65b-51ec-42e2-a4da-071eb613eac6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c"
Nov 04 03:24:36.080382 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:36.080365    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c"
Nov 04 03:24:36.534694 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:36.534653    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c"]
Nov 04 03:24:36.539209 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:24:36.539181    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod416ff65b_51ec_42e2_a4da_071eb613eac6.slice/crio-0c08270d9c3d39cb81e569aa60bd067f3e5aea83122540fb213f21627dc12ed9 WatchSource:0}: Error finding container 0c08270d9c3d39cb81e569aa60bd067f3e5aea83122540fb213f21627dc12ed9: Status 404 returned error can't find the container with id 0c08270d9c3d39cb81e569aa60bd067f3e5aea83122540fb213f21627dc12ed9
Nov 04 03:24:36.541439 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:36.541389    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:24:36.541512 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:36.541475    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:24:36.541556 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:36.541505    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:24:37.010841 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:37.010804    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c" event={"ID":"416ff65b-51ec-42e2-a4da-071eb613eac6","Type":"ContainerStarted","Data":"74f053dbc3792c096479b22caa517326a7324003a7f83e28942f0a9172491659"}
Nov 04 03:24:37.011328 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:37.010878    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c" event={"ID":"416ff65b-51ec-42e2-a4da-071eb613eac6","Type":"ContainerStarted","Data":"0c08270d9c3d39cb81e569aa60bd067f3e5aea83122540fb213f21627dc12ed9"}
Nov 04 03:24:37.081089 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:37.081049    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c"
Nov 04 03:24:37.085986 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:37.085961    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c"
Nov 04 03:24:37.127665 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:37.127601    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c" podStartSLOduration=2.127587636 podStartE2EDuration="2.127587636s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:24:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:24:37.125338658 +0000 UTC m=+419.701991572" watchObservedRunningTime="2026-11-04 03:24:37.127587636 +0000 UTC m=+419.704240550"
Nov 04 03:24:37.819351 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:37.819318    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-hvj2g"]
Nov 04 03:24:37.824284 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:37.824267    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-hvj2g"
Nov 04 03:24:37.875445 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:37.875411    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Nov 04 03:24:37.877919 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:37.877887    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-hvj2g"]
Nov 04 03:24:37.922453 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:37.922427    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-cfrc6\""
Nov 04 03:24:37.939452 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:37.939419    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/febfc9a5-a03f-4a67-a394-5e57ada3568c-config-file\") pod \"limitador-limitador-74dfcfb894-hvj2g\" (UID: \"febfc9a5-a03f-4a67-a394-5e57ada3568c\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-hvj2g"
Nov 04 03:24:37.939641 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:37.939458    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ln7kw\" (UniqueName: \"kubernetes.io/projected/febfc9a5-a03f-4a67-a394-5e57ada3568c-kube-api-access-ln7kw\") pod \"limitador-limitador-74dfcfb894-hvj2g\" (UID: \"febfc9a5-a03f-4a67-a394-5e57ada3568c\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-hvj2g"
Nov 04 03:24:38.016699 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:38.016670    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c"
Nov 04 03:24:38.017780 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:38.017758    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c"
Nov 04 03:24:38.039997 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:38.039963    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ln7kw\" (UniqueName: \"kubernetes.io/projected/febfc9a5-a03f-4a67-a394-5e57ada3568c-kube-api-access-ln7kw\") pod \"limitador-limitador-74dfcfb894-hvj2g\" (UID: \"febfc9a5-a03f-4a67-a394-5e57ada3568c\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-hvj2g"
Nov 04 03:24:38.040142 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:38.040123    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/febfc9a5-a03f-4a67-a394-5e57ada3568c-config-file\") pod \"limitador-limitador-74dfcfb894-hvj2g\" (UID: \"febfc9a5-a03f-4a67-a394-5e57ada3568c\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-hvj2g"
Nov 04 03:24:38.040974 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:38.040944    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/febfc9a5-a03f-4a67-a394-5e57ada3568c-config-file\") pod \"limitador-limitador-74dfcfb894-hvj2g\" (UID: \"febfc9a5-a03f-4a67-a394-5e57ada3568c\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-hvj2g"
Nov 04 03:24:38.042296 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:38.042273    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-hvj2g"]
Nov 04 03:24:38.114631 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:38.114561    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ln7kw\" (UniqueName: \"kubernetes.io/projected/febfc9a5-a03f-4a67-a394-5e57ada3568c-kube-api-access-ln7kw\") pod \"limitador-limitador-74dfcfb894-hvj2g\" (UID: \"febfc9a5-a03f-4a67-a394-5e57ada3568c\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-hvj2g"
Nov 04 03:24:38.137212 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:38.137173    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-hvj2g"
Nov 04 03:24:38.515037 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:38.515006    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-hvj2g"]
Nov 04 03:24:38.521065 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:24:38.521028    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfebfc9a5_a03f_4a67_a394_5e57ada3568c.slice/crio-100a7e519afce5c24bb7bdba2247da1f312fd935b3c00a613f7aaaaea48fc832 WatchSource:0}: Error finding container 100a7e519afce5c24bb7bdba2247da1f312fd935b3c00a613f7aaaaea48fc832: Status 404 returned error can't find the container with id 100a7e519afce5c24bb7bdba2247da1f312fd935b3c00a613f7aaaaea48fc832
Nov 04 03:24:38.992896 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:38.992860    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-57f7f8659f-mqhqz"]
Nov 04 03:24:38.997574 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:38.997554    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-mqhqz"
Nov 04 03:24:39.023066 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:39.023029    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-hvj2g" event={"ID":"febfc9a5-a03f-4a67-a394-5e57ada3568c","Type":"ContainerStarted","Data":"100a7e519afce5c24bb7bdba2247da1f312fd935b3c00a613f7aaaaea48fc832"}
Nov 04 03:24:39.047370 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:39.047337    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-trsqp\""
Nov 04 03:24:39.049452 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:39.049430    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-mqhqz"]
Nov 04 03:24:39.152655 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:39.152620    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pvfp\" (UniqueName: \"kubernetes.io/projected/d53ea49d-82e8-4495-87fe-685f9af7f2f6-kube-api-access-6pvfp\") pod \"authorino-57f7f8659f-mqhqz\" (UID: \"d53ea49d-82e8-4495-87fe-685f9af7f2f6\") " pod="kuadrant-system/authorino-57f7f8659f-mqhqz"
Nov 04 03:24:39.254219 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:39.254124    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pvfp\" (UniqueName: \"kubernetes.io/projected/d53ea49d-82e8-4495-87fe-685f9af7f2f6-kube-api-access-6pvfp\") pod \"authorino-57f7f8659f-mqhqz\" (UID: \"d53ea49d-82e8-4495-87fe-685f9af7f2f6\") " pod="kuadrant-system/authorino-57f7f8659f-mqhqz"
Nov 04 03:24:39.312911 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:39.312880    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pvfp\" (UniqueName: \"kubernetes.io/projected/d53ea49d-82e8-4495-87fe-685f9af7f2f6-kube-api-access-6pvfp\") pod \"authorino-57f7f8659f-mqhqz\" (UID: \"d53ea49d-82e8-4495-87fe-685f9af7f2f6\") " pod="kuadrant-system/authorino-57f7f8659f-mqhqz"
Nov 04 03:24:39.610230 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:39.610141    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-mqhqz"
Nov 04 03:24:39.986337 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:39.986298    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-mqhqz"]
Nov 04 03:24:39.992090 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:24:39.992064    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd53ea49d_82e8_4495_87fe_685f9af7f2f6.slice/crio-379e18e8731b0a5bae3ed3af2ff26cf719487cee9ca6d237e9a98bbde4380a1a WatchSource:0}: Error finding container 379e18e8731b0a5bae3ed3af2ff26cf719487cee9ca6d237e9a98bbde4380a1a: Status 404 returned error can't find the container with id 379e18e8731b0a5bae3ed3af2ff26cf719487cee9ca6d237e9a98bbde4380a1a
Nov 04 03:24:40.030124 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:40.030091    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-mqhqz" event={"ID":"d53ea49d-82e8-4495-87fe-685f9af7f2f6","Type":"ContainerStarted","Data":"379e18e8731b0a5bae3ed3af2ff26cf719487cee9ca6d237e9a98bbde4380a1a"}
Nov 04 03:24:42.048057 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:42.048016    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-hvj2g" event={"ID":"febfc9a5-a03f-4a67-a394-5e57ada3568c","Type":"ContainerStarted","Data":"f8080b9250e0214016dd051d1ef5eeecd21321e320236cc002a7e08d4bfd9943"}
Nov 04 03:24:42.048470 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:42.048176    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-hvj2g"
Nov 04 03:24:42.159491 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:42.159432    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-74dfcfb894-hvj2g" podStartSLOduration=2.697022895 podStartE2EDuration="5.159418772s" totalImagesPullingTime="2.462395877s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:24:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:24:42.158668575 +0000 UTC m=+424.735321491" watchObservedRunningTime="2026-11-04 03:24:42.159418772 +0000 UTC m=+424.736071683"
Nov 04 03:24:44.060964 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:44.060929    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-mqhqz" event={"ID":"d53ea49d-82e8-4495-87fe-685f9af7f2f6","Type":"ContainerStarted","Data":"b5b9416387ee52899af2b7125ae8c7483e9df02daa2181d8192dbafb4d100aa6"}
Nov 04 03:24:44.169806 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:44.169758    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-57f7f8659f-mqhqz" podStartSLOduration=2.779192745 podStartE2EDuration="6.16974401s" totalImagesPullingTime="3.390551265s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:24:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:24:44.168057829 +0000 UTC m=+426.744710743" watchObservedRunningTime="2026-11-04 03:24:44.16974401 +0000 UTC m=+426.746396946"
Nov 04 03:24:45.107943 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:45.107910    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-mqhqz"]
Nov 04 03:24:46.071458 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:46.071359    5609 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-57f7f8659f-mqhqz" podUID="d53ea49d-82e8-4495-87fe-685f9af7f2f6" containerName="authorino" containerID="cri-o://b5b9416387ee52899af2b7125ae8c7483e9df02daa2181d8192dbafb4d100aa6" gracePeriod=30
Nov 04 03:24:46.345541 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:46.345472    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-mqhqz"
Nov 04 03:24:46.522473 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:46.522436    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d53ea49d-82e8-4495-87fe-685f9af7f2f6-kube-api-access-6pvfp\" (UniqueName: \"kubernetes.io/projected/d53ea49d-82e8-4495-87fe-685f9af7f2f6-kube-api-access-6pvfp\") pod \"d53ea49d-82e8-4495-87fe-685f9af7f2f6\" (UID: \"d53ea49d-82e8-4495-87fe-685f9af7f2f6\") "
Nov 04 03:24:46.524604 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:46.524577    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d53ea49d-82e8-4495-87fe-685f9af7f2f6-kube-api-access-6pvfp" pod "d53ea49d-82e8-4495-87fe-685f9af7f2f6" (UID: "d53ea49d-82e8-4495-87fe-685f9af7f2f6"). InnerVolumeSpecName "kube-api-access-6pvfp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:24:46.623704 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:46.623607    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6pvfp\" (UniqueName: \"kubernetes.io/projected/d53ea49d-82e8-4495-87fe-685f9af7f2f6-kube-api-access-6pvfp\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:24:47.078335 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:47.078299    5609 generic.go:356] "Generic (PLEG): container finished" podID="d53ea49d-82e8-4495-87fe-685f9af7f2f6" containerID="b5b9416387ee52899af2b7125ae8c7483e9df02daa2181d8192dbafb4d100aa6" exitCode=0
Nov 04 03:24:47.078562 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:47.078348    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-mqhqz"
Nov 04 03:24:47.078562 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:47.078349    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-mqhqz" event={"ID":"d53ea49d-82e8-4495-87fe-685f9af7f2f6","Type":"ContainerDied","Data":"b5b9416387ee52899af2b7125ae8c7483e9df02daa2181d8192dbafb4d100aa6"}
Nov 04 03:24:47.078562 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:47.078478    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-mqhqz" event={"ID":"d53ea49d-82e8-4495-87fe-685f9af7f2f6","Type":"ContainerDied","Data":"379e18e8731b0a5bae3ed3af2ff26cf719487cee9ca6d237e9a98bbde4380a1a"}
Nov 04 03:24:47.078562 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:47.078511    5609 scope.go:122] "RemoveContainer" containerID="b5b9416387ee52899af2b7125ae8c7483e9df02daa2181d8192dbafb4d100aa6"
Nov 04 03:24:47.087899 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:47.087881    5609 scope.go:122] "RemoveContainer" containerID="b5b9416387ee52899af2b7125ae8c7483e9df02daa2181d8192dbafb4d100aa6"
Nov 04 03:24:47.088149 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:24:47.088129    5609 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5b9416387ee52899af2b7125ae8c7483e9df02daa2181d8192dbafb4d100aa6\": container with ID starting with b5b9416387ee52899af2b7125ae8c7483e9df02daa2181d8192dbafb4d100aa6 not found: ID does not exist" containerID="b5b9416387ee52899af2b7125ae8c7483e9df02daa2181d8192dbafb4d100aa6"
Nov 04 03:24:47.088209 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:47.088159    5609 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5b9416387ee52899af2b7125ae8c7483e9df02daa2181d8192dbafb4d100aa6"} err="failed to get container status \"b5b9416387ee52899af2b7125ae8c7483e9df02daa2181d8192dbafb4d100aa6\": rpc error: code = NotFound desc = could not find container \"b5b9416387ee52899af2b7125ae8c7483e9df02daa2181d8192dbafb4d100aa6\": container with ID starting with b5b9416387ee52899af2b7125ae8c7483e9df02daa2181d8192dbafb4d100aa6 not found: ID does not exist"
Nov 04 03:24:47.233225 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:47.233191    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-mqhqz"]
Nov 04 03:24:47.237689 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:47.237659    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-mqhqz"]
Nov 04 03:24:47.540061 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:47.540019    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d53ea49d-82e8-4495-87fe-685f9af7f2f6" path="/var/lib/kubelet/pods/d53ea49d-82e8-4495-87fe-685f9af7f2f6/volumes"
Nov 04 03:24:48.560157 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:24:48.560118    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:24:53.055202 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:24:53.055172    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-hvj2g"
Nov 04 03:25:00.273932 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:00.273890    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-qhqnk"]
Nov 04 03:25:00.274453 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:00.274424    5609 cpu_manager.go:414] "containerMap: removing container" podUID="d53ea49d-82e8-4495-87fe-685f9af7f2f6" containerName="authorino"
Nov 04 03:25:00.274453 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:00.274442    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d53ea49d-82e8-4495-87fe-685f9af7f2f6" containerName="authorino"
Nov 04 03:25:00.274617 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:00.274600    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="d53ea49d-82e8-4495-87fe-685f9af7f2f6" containerName="authorino"
Nov 04 03:25:00.277125 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:00.277103    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-qhqnk"
Nov 04 03:25:00.326912 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:00.326883    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Nov 04 03:25:00.327020 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:00.326925    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-module-controller-manager-dockercfg-75ktv\""
Nov 04 03:25:00.327020 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:00.326964    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Nov 04 03:25:00.343308 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:00.343285    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/bb2a6c09-2ea4-407c-985f-5dfa99b191e3-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-qhqnk\" (UID: \"bb2a6c09-2ea4-407c-985f-5dfa99b191e3\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-qhqnk"
Nov 04 03:25:00.343437 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:00.343328    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb2a6c09-2ea4-407c-985f-5dfa99b191e3-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-qhqnk\" (UID: \"bb2a6c09-2ea4-407c-985f-5dfa99b191e3\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-qhqnk"
Nov 04 03:25:00.343484 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:00.343443    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdpl5\" (UniqueName: \"kubernetes.io/projected/bb2a6c09-2ea4-407c-985f-5dfa99b191e3-kube-api-access-bdpl5\") pod \"kserve-module-controller-manager-7566cf5fc8-qhqnk\" (UID: \"bb2a6c09-2ea4-407c-985f-5dfa99b191e3\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-qhqnk"
Nov 04 03:25:00.343524 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:00.343486    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bb2a6c09-2ea4-407c-985f-5dfa99b191e3-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-qhqnk\" (UID: \"bb2a6c09-2ea4-407c-985f-5dfa99b191e3\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-qhqnk"
Nov 04 03:25:00.444083 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:00.444053    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/bb2a6c09-2ea4-407c-985f-5dfa99b191e3-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-qhqnk\" (UID: \"bb2a6c09-2ea4-407c-985f-5dfa99b191e3\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-qhqnk"
Nov 04 03:25:00.444248 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:00.444096    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb2a6c09-2ea4-407c-985f-5dfa99b191e3-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-qhqnk\" (UID: \"bb2a6c09-2ea4-407c-985f-5dfa99b191e3\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-qhqnk"
Nov 04 03:25:00.444248 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:00.444126    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdpl5\" (UniqueName: \"kubernetes.io/projected/bb2a6c09-2ea4-407c-985f-5dfa99b191e3-kube-api-access-bdpl5\") pod \"kserve-module-controller-manager-7566cf5fc8-qhqnk\" (UID: \"bb2a6c09-2ea4-407c-985f-5dfa99b191e3\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-qhqnk"
Nov 04 03:25:00.444338 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:00.444244    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bb2a6c09-2ea4-407c-985f-5dfa99b191e3-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-qhqnk\" (UID: \"bb2a6c09-2ea4-407c-985f-5dfa99b191e3\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-qhqnk"
Nov 04 03:25:00.444473 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:00.444455    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/bb2a6c09-2ea4-407c-985f-5dfa99b191e3-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-qhqnk\" (UID: \"bb2a6c09-2ea4-407c-985f-5dfa99b191e3\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-qhqnk"
Nov 04 03:25:00.444539 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:00.444522    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb2a6c09-2ea4-407c-985f-5dfa99b191e3-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-qhqnk\" (UID: \"bb2a6c09-2ea4-407c-985f-5dfa99b191e3\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-qhqnk"
Nov 04 03:25:00.468632 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:00.468606    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"odh-trusted-ca-bundle\""
Nov 04 03:25:00.471218 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:00.471196    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-qhqnk"]
Nov 04 03:25:00.475414 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:00.475386    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bb2a6c09-2ea4-407c-985f-5dfa99b191e3-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-qhqnk\" (UID: \"bb2a6c09-2ea4-407c-985f-5dfa99b191e3\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-qhqnk"
Nov 04 03:25:00.502544 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:00.502516    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdpl5\" (UniqueName: \"kubernetes.io/projected/bb2a6c09-2ea4-407c-985f-5dfa99b191e3-kube-api-access-bdpl5\") pod \"kserve-module-controller-manager-7566cf5fc8-qhqnk\" (UID: \"bb2a6c09-2ea4-407c-985f-5dfa99b191e3\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-qhqnk"
Nov 04 03:25:00.589181 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:00.589107    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-qhqnk"
Nov 04 03:25:01.023141 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:01.023111    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-qhqnk"]
Nov 04 03:25:01.028226 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:25:01.028200    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb2a6c09_2ea4_407c_985f_5dfa99b191e3.slice/crio-02e5b451f4aecd6a86a4068d7c8f6afa873fc669cf94d4a96f118faa0fa0a50a WatchSource:0}: Error finding container 02e5b451f4aecd6a86a4068d7c8f6afa873fc669cf94d4a96f118faa0fa0a50a: Status 404 returned error can't find the container with id 02e5b451f4aecd6a86a4068d7c8f6afa873fc669cf94d4a96f118faa0fa0a50a
Nov 04 03:25:01.152729 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:01.152690    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-qhqnk" event={"ID":"bb2a6c09-2ea4-407c-985f-5dfa99b191e3","Type":"ContainerStarted","Data":"02e5b451f4aecd6a86a4068d7c8f6afa873fc669cf94d4a96f118faa0fa0a50a"}
Nov 04 03:25:04.173384 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:04.173346    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-qhqnk" event={"ID":"bb2a6c09-2ea4-407c-985f-5dfa99b191e3","Type":"ContainerStarted","Data":"c582d0b21d38400246a66dff905c1aa1c4f66e1fb0daf840774d857aa70a3360"}
Nov 04 03:25:04.173950 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:04.173572    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-qhqnk"
Nov 04 03:25:04.335431 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:04.335312    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-qhqnk" podStartSLOduration=1.412484992 podStartE2EDuration="4.335294813s" totalImagesPullingTime="2.922809821s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:25:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:25:04.333158578 +0000 UTC m=+446.909811495" watchObservedRunningTime="2026-11-04 03:25:04.335294813 +0000 UTC m=+446.911947731"
Nov 04 03:25:06.159935 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:06.159897    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-hvj2g"]
Nov 04 03:25:06.160460 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:06.160112    5609 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-74dfcfb894-hvj2g" podUID="febfc9a5-a03f-4a67-a394-5e57ada3568c" containerName="limitador" containerID="cri-o://f8080b9250e0214016dd051d1ef5eeecd21321e320236cc002a7e08d4bfd9943" gracePeriod=30
Nov 04 03:25:06.471353 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:25:06.471324    5609 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfebfc9a5_a03f_4a67_a394_5e57ada3568c.slice/crio-f8080b9250e0214016dd051d1ef5eeecd21321e320236cc002a7e08d4bfd9943.scope\": RecentStats: unable to find data in memory cache]"
Nov 04 03:25:06.714499 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:06.714476    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-hvj2g"
Nov 04 03:25:06.803651 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:06.803574    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/febfc9a5-a03f-4a67-a394-5e57ada3568c-kube-api-access-ln7kw\" (UniqueName: \"kubernetes.io/projected/febfc9a5-a03f-4a67-a394-5e57ada3568c-kube-api-access-ln7kw\") pod \"febfc9a5-a03f-4a67-a394-5e57ada3568c\" (UID: \"febfc9a5-a03f-4a67-a394-5e57ada3568c\") "
Nov 04 03:25:06.803798 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:06.803685    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/febfc9a5-a03f-4a67-a394-5e57ada3568c-config-file\" (UniqueName: \"kubernetes.io/configmap/febfc9a5-a03f-4a67-a394-5e57ada3568c-config-file\") pod \"febfc9a5-a03f-4a67-a394-5e57ada3568c\" (UID: \"febfc9a5-a03f-4a67-a394-5e57ada3568c\") "
Nov 04 03:25:06.804008 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:06.803985    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/febfc9a5-a03f-4a67-a394-5e57ada3568c-config-file" pod "febfc9a5-a03f-4a67-a394-5e57ada3568c" (UID: "febfc9a5-a03f-4a67-a394-5e57ada3568c"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:25:06.805742 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:06.805718    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/febfc9a5-a03f-4a67-a394-5e57ada3568c-kube-api-access-ln7kw" pod "febfc9a5-a03f-4a67-a394-5e57ada3568c" (UID: "febfc9a5-a03f-4a67-a394-5e57ada3568c"). InnerVolumeSpecName "kube-api-access-ln7kw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:25:06.904248 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:06.904210    5609 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/febfc9a5-a03f-4a67-a394-5e57ada3568c-config-file\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:25:06.904248 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:06.904246    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ln7kw\" (UniqueName: \"kubernetes.io/projected/febfc9a5-a03f-4a67-a394-5e57ada3568c-kube-api-access-ln7kw\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:25:07.191504 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:07.191472    5609 generic.go:356] "Generic (PLEG): container finished" podID="febfc9a5-a03f-4a67-a394-5e57ada3568c" containerID="f8080b9250e0214016dd051d1ef5eeecd21321e320236cc002a7e08d4bfd9943" exitCode=0
Nov 04 03:25:07.191922 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:07.191522    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-hvj2g"
Nov 04 03:25:07.191922 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:07.191541    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-hvj2g" event={"ID":"febfc9a5-a03f-4a67-a394-5e57ada3568c","Type":"ContainerDied","Data":"f8080b9250e0214016dd051d1ef5eeecd21321e320236cc002a7e08d4bfd9943"}
Nov 04 03:25:07.191922 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:07.191606    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-hvj2g" event={"ID":"febfc9a5-a03f-4a67-a394-5e57ada3568c","Type":"ContainerDied","Data":"100a7e519afce5c24bb7bdba2247da1f312fd935b3c00a613f7aaaaea48fc832"}
Nov 04 03:25:07.191922 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:07.191627    5609 scope.go:122] "RemoveContainer" containerID="f8080b9250e0214016dd051d1ef5eeecd21321e320236cc002a7e08d4bfd9943"
Nov 04 03:25:07.200796 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:07.200772    5609 scope.go:122] "RemoveContainer" containerID="f8080b9250e0214016dd051d1ef5eeecd21321e320236cc002a7e08d4bfd9943"
Nov 04 03:25:07.201084 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:25:07.201064    5609 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8080b9250e0214016dd051d1ef5eeecd21321e320236cc002a7e08d4bfd9943\": container with ID starting with f8080b9250e0214016dd051d1ef5eeecd21321e320236cc002a7e08d4bfd9943 not found: ID does not exist" containerID="f8080b9250e0214016dd051d1ef5eeecd21321e320236cc002a7e08d4bfd9943"
Nov 04 03:25:07.201135 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:07.201093    5609 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8080b9250e0214016dd051d1ef5eeecd21321e320236cc002a7e08d4bfd9943"} err="failed to get container status \"f8080b9250e0214016dd051d1ef5eeecd21321e320236cc002a7e08d4bfd9943\": rpc error: code = NotFound desc = could not find container \"f8080b9250e0214016dd051d1ef5eeecd21321e320236cc002a7e08d4bfd9943\": container with ID starting with f8080b9250e0214016dd051d1ef5eeecd21321e320236cc002a7e08d4bfd9943 not found: ID does not exist"
Nov 04 03:25:07.347015 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:07.346976    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-hvj2g"]
Nov 04 03:25:07.394329 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:07.394292    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-hvj2g"]
Nov 04 03:25:07.533107 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:07.533032    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="febfc9a5-a03f-4a67-a394-5e57ada3568c" path="/var/lib/kubelet/pods/febfc9a5-a03f-4a67-a394-5e57ada3568c/volumes"
Nov 04 03:25:16.627332 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:16.627293    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-mp4fd"]
Nov 04 03:25:16.627740 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:16.627673    5609 cpu_manager.go:414] "containerMap: removing container" podUID="febfc9a5-a03f-4a67-a394-5e57ada3568c" containerName="limitador"
Nov 04 03:25:16.627740 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:16.627685    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="febfc9a5-a03f-4a67-a394-5e57ada3568c" containerName="limitador"
Nov 04 03:25:16.627821 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:16.627765    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="febfc9a5-a03f-4a67-a394-5e57ada3568c" containerName="limitador"
Nov 04 03:25:16.633850 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:16.633828    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-mp4fd"
Nov 04 03:25:16.685929 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:16.685891    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-mp4fd"]
Nov 04 03:25:16.730536 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:16.730495    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Nov 04 03:25:16.730536 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:16.730502    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 03:25:16.730536 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:16.730540    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 03:25:16.730804 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:16.730565    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-xjl9g\""
Nov 04 03:25:16.786691 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:16.786653    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gln6w\" (UniqueName: \"kubernetes.io/projected/4db3855d-8b8d-4b57-a83f-1a8da42cda50-kube-api-access-gln6w\") pod \"postgres-7cdbf6dc44-mp4fd\" (UID: \"4db3855d-8b8d-4b57-a83f-1a8da42cda50\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-mp4fd"
Nov 04 03:25:16.786899 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:16.786750    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4db3855d-8b8d-4b57-a83f-1a8da42cda50-data\") pod \"postgres-7cdbf6dc44-mp4fd\" (UID: \"4db3855d-8b8d-4b57-a83f-1a8da42cda50\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-mp4fd"
Nov 04 03:25:16.887750 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:16.887658    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gln6w\" (UniqueName: \"kubernetes.io/projected/4db3855d-8b8d-4b57-a83f-1a8da42cda50-kube-api-access-gln6w\") pod \"postgres-7cdbf6dc44-mp4fd\" (UID: \"4db3855d-8b8d-4b57-a83f-1a8da42cda50\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-mp4fd"
Nov 04 03:25:16.887750 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:16.887705    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4db3855d-8b8d-4b57-a83f-1a8da42cda50-data\") pod \"postgres-7cdbf6dc44-mp4fd\" (UID: \"4db3855d-8b8d-4b57-a83f-1a8da42cda50\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-mp4fd"
Nov 04 03:25:16.888096 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:16.888080    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4db3855d-8b8d-4b57-a83f-1a8da42cda50-data\") pod \"postgres-7cdbf6dc44-mp4fd\" (UID: \"4db3855d-8b8d-4b57-a83f-1a8da42cda50\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-mp4fd"
Nov 04 03:25:16.946023 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:16.945987    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gln6w\" (UniqueName: \"kubernetes.io/projected/4db3855d-8b8d-4b57-a83f-1a8da42cda50-kube-api-access-gln6w\") pod \"postgres-7cdbf6dc44-mp4fd\" (UID: \"4db3855d-8b8d-4b57-a83f-1a8da42cda50\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-mp4fd"
Nov 04 03:25:16.947820 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:16.947799    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-mp4fd"
Nov 04 03:25:17.321276 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:17.321238    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-mp4fd"]
Nov 04 03:25:17.326995 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:25:17.326953    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4db3855d_8b8d_4b57_a83f_1a8da42cda50.slice/crio-e884c43f616101b5d54099fd5bd8af380e7d00218cc714a9c395e62feadb3866 WatchSource:0}: Error finding container e884c43f616101b5d54099fd5bd8af380e7d00218cc714a9c395e62feadb3866: Status 404 returned error can't find the container with id e884c43f616101b5d54099fd5bd8af380e7d00218cc714a9c395e62feadb3866
Nov 04 03:25:18.253593 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:18.253557    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-mp4fd" event={"ID":"4db3855d-8b8d-4b57-a83f-1a8da42cda50","Type":"ContainerStarted","Data":"e884c43f616101b5d54099fd5bd8af380e7d00218cc714a9c395e62feadb3866"}
Nov 04 03:25:18.559225 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:25:18.559144    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:25:25.181954 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:25.181930    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-qhqnk"
Nov 04 03:25:25.296362 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:25.296328    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-mp4fd" event={"ID":"4db3855d-8b8d-4b57-a83f-1a8da42cda50","Type":"ContainerStarted","Data":"35ad1cc06360992fbc87cf8c360a314f60eccef55618587a4555c39a77f28212"}
Nov 04 03:25:25.296542 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:25.296388    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-mp4fd"
Nov 04 03:25:25.415554 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:25.415466    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-mp4fd" podStartSLOduration=1.6133424 podStartE2EDuration="9.415451608s" totalImagesPullingTime="7.802109208s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:25:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:25:25.41350203 +0000 UTC m=+467.990154945" watchObservedRunningTime="2026-11-04 03:25:25.415451608 +0000 UTC m=+467.992104521"
Nov 04 03:25:31.331082 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:31.331053    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-mp4fd"
Nov 04 03:25:38.512247 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:38.512206    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-589f8697fb-5r6hk"]
Nov 04 03:25:38.600772 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:38.600742    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-589f8697fb-5r6hk"]
Nov 04 03:25:38.600962 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:38.600870    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-589f8697fb-5r6hk"
Nov 04 03:25:38.650217 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:38.650177    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Nov 04 03:25:38.697328 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:38.697295    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Nov 04 03:25:38.697328 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:38.697299    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-4zdr5\""
Nov 04 03:25:38.697627 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:38.697504    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Nov 04 03:25:38.784347 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:38.784246    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/47389e83-5848-4b0c-9d49-6681582c6ede-webhook-cert\") pod \"maas-controller-589f8697fb-5r6hk\" (UID: \"47389e83-5848-4b0c-9d49-6681582c6ede\") " pod="opendatahub/maas-controller-589f8697fb-5r6hk"
Nov 04 03:25:38.784347 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:38.784318    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clmqx\" (UniqueName: \"kubernetes.io/projected/47389e83-5848-4b0c-9d49-6681582c6ede-kube-api-access-clmqx\") pod \"maas-controller-589f8697fb-5r6hk\" (UID: \"47389e83-5848-4b0c-9d49-6681582c6ede\") " pod="opendatahub/maas-controller-589f8697fb-5r6hk"
Nov 04 03:25:38.784593 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:38.784529    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47389e83-5848-4b0c-9d49-6681582c6ede-metrics-certs\") pod \"maas-controller-589f8697fb-5r6hk\" (UID: \"47389e83-5848-4b0c-9d49-6681582c6ede\") " pod="opendatahub/maas-controller-589f8697fb-5r6hk"
Nov 04 03:25:38.885540 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:38.885505    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clmqx\" (UniqueName: \"kubernetes.io/projected/47389e83-5848-4b0c-9d49-6681582c6ede-kube-api-access-clmqx\") pod \"maas-controller-589f8697fb-5r6hk\" (UID: \"47389e83-5848-4b0c-9d49-6681582c6ede\") " pod="opendatahub/maas-controller-589f8697fb-5r6hk"
Nov 04 03:25:38.885764 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:38.885576    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47389e83-5848-4b0c-9d49-6681582c6ede-metrics-certs\") pod \"maas-controller-589f8697fb-5r6hk\" (UID: \"47389e83-5848-4b0c-9d49-6681582c6ede\") " pod="opendatahub/maas-controller-589f8697fb-5r6hk"
Nov 04 03:25:38.885764 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:38.885618    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/47389e83-5848-4b0c-9d49-6681582c6ede-webhook-cert\") pod \"maas-controller-589f8697fb-5r6hk\" (UID: \"47389e83-5848-4b0c-9d49-6681582c6ede\") " pod="opendatahub/maas-controller-589f8697fb-5r6hk"
Nov 04 03:25:38.888261 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:38.888228    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/47389e83-5848-4b0c-9d49-6681582c6ede-webhook-cert\") pod \"maas-controller-589f8697fb-5r6hk\" (UID: \"47389e83-5848-4b0c-9d49-6681582c6ede\") " pod="opendatahub/maas-controller-589f8697fb-5r6hk"
Nov 04 03:25:38.888383 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:38.888244    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47389e83-5848-4b0c-9d49-6681582c6ede-metrics-certs\") pod \"maas-controller-589f8697fb-5r6hk\" (UID: \"47389e83-5848-4b0c-9d49-6681582c6ede\") " pod="opendatahub/maas-controller-589f8697fb-5r6hk"
Nov 04 03:25:38.943879 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:38.943851    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-clmqx\" (UniqueName: \"kubernetes.io/projected/47389e83-5848-4b0c-9d49-6681582c6ede-kube-api-access-clmqx\") pod \"maas-controller-589f8697fb-5r6hk\" (UID: \"47389e83-5848-4b0c-9d49-6681582c6ede\") " pod="opendatahub/maas-controller-589f8697fb-5r6hk"
Nov 04 03:25:39.085613 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:39.085515    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5fb896bc89-fkk4g"]
Nov 04 03:25:39.090148 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:39.090122    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5fb896bc89-fkk4g"
Nov 04 03:25:39.147472 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:39.146787    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5fb896bc89-fkk4g"]
Nov 04 03:25:39.188386 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:39.188350    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6af8c02-22bc-49e1-adb9-2f01da0af47c-metrics-certs\") pod \"maas-controller-5fb896bc89-fkk4g\" (UID: \"e6af8c02-22bc-49e1-adb9-2f01da0af47c\") " pod="opendatahub/maas-controller-5fb896bc89-fkk4g"
Nov 04 03:25:39.188606 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:39.188430    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e6af8c02-22bc-49e1-adb9-2f01da0af47c-webhook-cert\") pod \"maas-controller-5fb896bc89-fkk4g\" (UID: \"e6af8c02-22bc-49e1-adb9-2f01da0af47c\") " pod="opendatahub/maas-controller-5fb896bc89-fkk4g"
Nov 04 03:25:39.188606 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:39.188571    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ft8l\" (UniqueName: \"kubernetes.io/projected/e6af8c02-22bc-49e1-adb9-2f01da0af47c-kube-api-access-7ft8l\") pod \"maas-controller-5fb896bc89-fkk4g\" (UID: \"e6af8c02-22bc-49e1-adb9-2f01da0af47c\") " pod="opendatahub/maas-controller-5fb896bc89-fkk4g"
Nov 04 03:25:39.227805 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:39.227765    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-589f8697fb-5r6hk"
Nov 04 03:25:39.302425 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:39.301310    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6af8c02-22bc-49e1-adb9-2f01da0af47c-metrics-certs\") pod \"maas-controller-5fb896bc89-fkk4g\" (UID: \"e6af8c02-22bc-49e1-adb9-2f01da0af47c\") " pod="opendatahub/maas-controller-5fb896bc89-fkk4g"
Nov 04 03:25:39.302425 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:39.301387    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e6af8c02-22bc-49e1-adb9-2f01da0af47c-webhook-cert\") pod \"maas-controller-5fb896bc89-fkk4g\" (UID: \"e6af8c02-22bc-49e1-adb9-2f01da0af47c\") " pod="opendatahub/maas-controller-5fb896bc89-fkk4g"
Nov 04 03:25:39.302425 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:39.301575    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ft8l\" (UniqueName: \"kubernetes.io/projected/e6af8c02-22bc-49e1-adb9-2f01da0af47c-kube-api-access-7ft8l\") pod \"maas-controller-5fb896bc89-fkk4g\" (UID: \"e6af8c02-22bc-49e1-adb9-2f01da0af47c\") " pod="opendatahub/maas-controller-5fb896bc89-fkk4g"
Nov 04 03:25:39.305035 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:39.305004    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6af8c02-22bc-49e1-adb9-2f01da0af47c-metrics-certs\") pod \"maas-controller-5fb896bc89-fkk4g\" (UID: \"e6af8c02-22bc-49e1-adb9-2f01da0af47c\") " pod="opendatahub/maas-controller-5fb896bc89-fkk4g"
Nov 04 03:25:39.305938 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:39.305893    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e6af8c02-22bc-49e1-adb9-2f01da0af47c-webhook-cert\") pod \"maas-controller-5fb896bc89-fkk4g\" (UID: \"e6af8c02-22bc-49e1-adb9-2f01da0af47c\") " pod="opendatahub/maas-controller-5fb896bc89-fkk4g"
Nov 04 03:25:39.360188 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:39.360104    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ft8l\" (UniqueName: \"kubernetes.io/projected/e6af8c02-22bc-49e1-adb9-2f01da0af47c-kube-api-access-7ft8l\") pod \"maas-controller-5fb896bc89-fkk4g\" (UID: \"e6af8c02-22bc-49e1-adb9-2f01da0af47c\") " pod="opendatahub/maas-controller-5fb896bc89-fkk4g"
Nov 04 03:25:39.402925 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:39.402882    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5fb896bc89-fkk4g"
Nov 04 03:25:39.601795 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:39.601757    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-589f8697fb-5r6hk"]
Nov 04 03:25:39.608839 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:25:39.608793    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47389e83_5848_4b0c_9d49_6681582c6ede.slice/crio-79bb0f4c39380fab5d531d13d6ea8aba271bdf58c594761d24cb6d76aa56a608 WatchSource:0}: Error finding container 79bb0f4c39380fab5d531d13d6ea8aba271bdf58c594761d24cb6d76aa56a608: Status 404 returned error can't find the container with id 79bb0f4c39380fab5d531d13d6ea8aba271bdf58c594761d24cb6d76aa56a608
Nov 04 03:25:39.776948 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:39.776912    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5fb896bc89-fkk4g"]
Nov 04 03:25:39.782236 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:25:39.782209    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6af8c02_22bc_49e1_adb9_2f01da0af47c.slice/crio-3d8c229c77c2af6f2ba2d55f4d24566e19f08c5ad95d2feb58ce4230f0dacf93 WatchSource:0}: Error finding container 3d8c229c77c2af6f2ba2d55f4d24566e19f08c5ad95d2feb58ce4230f0dacf93: Status 404 returned error can't find the container with id 3d8c229c77c2af6f2ba2d55f4d24566e19f08c5ad95d2feb58ce4230f0dacf93
Nov 04 03:25:40.376651 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:40.376612    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-589f8697fb-5r6hk" event={"ID":"47389e83-5848-4b0c-9d49-6681582c6ede","Type":"ContainerStarted","Data":"79bb0f4c39380fab5d531d13d6ea8aba271bdf58c594761d24cb6d76aa56a608"}
Nov 04 03:25:40.377965 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:40.377936    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5fb896bc89-fkk4g" event={"ID":"e6af8c02-22bc-49e1-adb9-2f01da0af47c","Type":"ContainerStarted","Data":"3d8c229c77c2af6f2ba2d55f4d24566e19f08c5ad95d2feb58ce4230f0dacf93"}
Nov 04 03:25:47.424069 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:47.424017    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5fb896bc89-fkk4g" event={"ID":"e6af8c02-22bc-49e1-adb9-2f01da0af47c","Type":"ContainerStarted","Data":"509dcec1e5785ea67375bc0593810695134495c9bc5c13bae31f47e52dc68238"}
Nov 04 03:25:47.424570 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:47.424125    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5fb896bc89-fkk4g"
Nov 04 03:25:47.425813 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:47.425780    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-589f8697fb-5r6hk" event={"ID":"47389e83-5848-4b0c-9d49-6681582c6ede","Type":"ContainerStarted","Data":"4337ffbdecae153d64dd8a49fa99528b5d63cd6120190ec0e10175de569f9ea1"}
Nov 04 03:25:47.425943 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:47.425846    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-589f8697fb-5r6hk"
Nov 04 03:25:47.546939 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:47.546875    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-5fb896bc89-fkk4g" podStartSLOduration=1.7508718170000002 podStartE2EDuration="8.546861511s" totalImagesPullingTime="6.795989694s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:25:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:25:47.544611751 +0000 UTC m=+490.121264665" watchObservedRunningTime="2026-11-04 03:25:47.546861511 +0000 UTC m=+490.123514455"
Nov 04 03:25:47.658296 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:47.658243    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-589f8697fb-5r6hk" podStartSLOduration=2.679113809 podStartE2EDuration="9.65822941s" totalImagesPullingTime="6.979115601s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:25:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:25:47.656518542 +0000 UTC m=+490.233171464" watchObservedRunningTime="2026-11-04 03:25:47.65822941 +0000 UTC m=+490.234882324"
Nov 04 03:25:48.559825 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:25:48.559788    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:25:50.017474 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:50.017437    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-98hr6"]
Nov 04 03:25:50.020739 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:50.020712    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-98hr6"
Nov 04 03:25:50.072359 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:50.072318    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-source\""
Nov 04 03:25:50.091123 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:50.091092    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c8183d45-a9c7-4fe4-bbfa-a1ddc7e33423-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-98hr6\" (UID: \"c8183d45-a9c7-4fe4-bbfa-a1ddc7e33423\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-98hr6"
Nov 04 03:25:50.091295 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:50.091160    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8pvq\" (UniqueName: \"kubernetes.io/projected/c8183d45-a9c7-4fe4-bbfa-a1ddc7e33423-kube-api-access-g8pvq\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-98hr6\" (UID: \"c8183d45-a9c7-4fe4-bbfa-a1ddc7e33423\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-98hr6"
Nov 04 03:25:50.091295 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:50.091256    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/c8183d45-a9c7-4fe4-bbfa-a1ddc7e33423-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-98hr6\" (UID: \"c8183d45-a9c7-4fe4-bbfa-a1ddc7e33423\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-98hr6"
Nov 04 03:25:50.091295 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:50.091284    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c8183d45-a9c7-4fe4-bbfa-a1ddc7e33423-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-98hr6\" (UID: \"c8183d45-a9c7-4fe4-bbfa-a1ddc7e33423\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-98hr6"
Nov 04 03:25:50.119738 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:50.119704    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-ca-bundle\""
Nov 04 03:25:50.119738 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:50.119703    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-tls\""
Nov 04 03:25:50.119965 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:50.119853    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-dockercfg-76nrl\""
Nov 04 03:25:50.122470 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:50.122447    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-98hr6"]
Nov 04 03:25:50.192204 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:50.192169    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8pvq\" (UniqueName: \"kubernetes.io/projected/c8183d45-a9c7-4fe4-bbfa-a1ddc7e33423-kube-api-access-g8pvq\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-98hr6\" (UID: \"c8183d45-a9c7-4fe4-bbfa-a1ddc7e33423\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-98hr6"
Nov 04 03:25:50.192395 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:50.192222    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/c8183d45-a9c7-4fe4-bbfa-a1ddc7e33423-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-98hr6\" (UID: \"c8183d45-a9c7-4fe4-bbfa-a1ddc7e33423\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-98hr6"
Nov 04 03:25:50.192395 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:50.192241    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c8183d45-a9c7-4fe4-bbfa-a1ddc7e33423-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-98hr6\" (UID: \"c8183d45-a9c7-4fe4-bbfa-a1ddc7e33423\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-98hr6"
Nov 04 03:25:50.192395 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:50.192282    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c8183d45-a9c7-4fe4-bbfa-a1ddc7e33423-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-98hr6\" (UID: \"c8183d45-a9c7-4fe4-bbfa-a1ddc7e33423\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-98hr6"
Nov 04 03:25:50.192395 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:25:50.192319    5609 configmap.go:193] Couldn't get configMap opendatahub/usage-tenancy-proxy-source: configmap "usage-tenancy-proxy-source" not found
Nov 04 03:25:50.192639 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:25:50.192417    5609 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c8183d45-a9c7-4fe4-bbfa-a1ddc7e33423-proxy-source podName:c8183d45-a9c7-4fe4-bbfa-a1ddc7e33423 nodeName:}" failed. No retries permitted until 2026-11-04 03:25:50.692380607 +0000 UTC m=+493.269033516 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-source" (UniqueName: "kubernetes.io/configmap/c8183d45-a9c7-4fe4-bbfa-a1ddc7e33423-proxy-source") pod "usage-logs-tenancy-proxy-9c5d49bcd-98hr6" (UID: "c8183d45-a9c7-4fe4-bbfa-a1ddc7e33423") : configmap "usage-tenancy-proxy-source" not found
Nov 04 03:25:50.192639 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:25:50.192420    5609 secret.go:189] Couldn't get secret opendatahub/usage-logs-tenancy-proxy-tls: secret "usage-logs-tenancy-proxy-tls" not found
Nov 04 03:25:50.192639 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:25:50.192485    5609 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8183d45-a9c7-4fe4-bbfa-a1ddc7e33423-tls-cert podName:c8183d45-a9c7-4fe4-bbfa-a1ddc7e33423 nodeName:}" failed. No retries permitted until 2026-11-04 03:25:50.692468317 +0000 UTC m=+493.269121225 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-cert" (UniqueName: "kubernetes.io/secret/c8183d45-a9c7-4fe4-bbfa-a1ddc7e33423-tls-cert") pod "usage-logs-tenancy-proxy-9c5d49bcd-98hr6" (UID: "c8183d45-a9c7-4fe4-bbfa-a1ddc7e33423") : secret "usage-logs-tenancy-proxy-tls" not found
Nov 04 03:25:50.192934 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:50.192916    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c8183d45-a9c7-4fe4-bbfa-a1ddc7e33423-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-98hr6\" (UID: \"c8183d45-a9c7-4fe4-bbfa-a1ddc7e33423\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-98hr6"
Nov 04 03:25:50.251384 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:50.251347    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8pvq\" (UniqueName: \"kubernetes.io/projected/c8183d45-a9c7-4fe4-bbfa-a1ddc7e33423-kube-api-access-g8pvq\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-98hr6\" (UID: \"c8183d45-a9c7-4fe4-bbfa-a1ddc7e33423\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-98hr6"
Nov 04 03:25:50.697234 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:50.697196    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/c8183d45-a9c7-4fe4-bbfa-a1ddc7e33423-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-98hr6\" (UID: \"c8183d45-a9c7-4fe4-bbfa-a1ddc7e33423\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-98hr6"
Nov 04 03:25:50.697234 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:50.697235    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c8183d45-a9c7-4fe4-bbfa-a1ddc7e33423-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-98hr6\" (UID: \"c8183d45-a9c7-4fe4-bbfa-a1ddc7e33423\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-98hr6"
Nov 04 03:25:50.698077 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:50.698055    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/c8183d45-a9c7-4fe4-bbfa-a1ddc7e33423-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-98hr6\" (UID: \"c8183d45-a9c7-4fe4-bbfa-a1ddc7e33423\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-98hr6"
Nov 04 03:25:50.699619 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:50.699600    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c8183d45-a9c7-4fe4-bbfa-a1ddc7e33423-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-98hr6\" (UID: \"c8183d45-a9c7-4fe4-bbfa-a1ddc7e33423\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-98hr6"
Nov 04 03:25:50.934010 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:50.933974    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-98hr6"
Nov 04 03:25:51.308114 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:51.308013    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-98hr6"]
Nov 04 03:25:51.319376 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:25:51.319341    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8183d45_a9c7_4fe4_bbfa_a1ddc7e33423.slice/crio-8f517837aa6f6bbd78d2d15ce142016ea56095d222f35788a21a25bff1512bd7 WatchSource:0}: Error finding container 8f517837aa6f6bbd78d2d15ce142016ea56095d222f35788a21a25bff1512bd7: Status 404 returned error can't find the container with id 8f517837aa6f6bbd78d2d15ce142016ea56095d222f35788a21a25bff1512bd7
Nov 04 03:25:51.448527 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:51.448485    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-98hr6" event={"ID":"c8183d45-a9c7-4fe4-bbfa-a1ddc7e33423","Type":"ContainerStarted","Data":"8f517837aa6f6bbd78d2d15ce142016ea56095d222f35788a21a25bff1512bd7"}
Nov 04 03:25:58.434693 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:58.434658    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5fb896bc89-fkk4g"
Nov 04 03:25:58.435272 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:58.434743    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-589f8697fb-5r6hk"
Nov 04 03:25:58.603990 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:58.603954    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-589f8697fb-5r6hk"]
Nov 04 03:25:58.604192 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:58.604163    5609 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-589f8697fb-5r6hk" podUID="47389e83-5848-4b0c-9d49-6681582c6ede" containerName="manager" containerID="cri-o://4337ffbdecae153d64dd8a49fa99528b5d63cd6120190ec0e10175de569f9ea1" gracePeriod=30
Nov 04 03:25:58.994379 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:58.994349    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-589f8697fb-5r6hk"
Nov 04 03:25:59.079504 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:59.079460    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/47389e83-5848-4b0c-9d49-6681582c6ede-kube-api-access-clmqx\" (UniqueName: \"kubernetes.io/projected/47389e83-5848-4b0c-9d49-6681582c6ede-kube-api-access-clmqx\") pod \"47389e83-5848-4b0c-9d49-6681582c6ede\" (UID: \"47389e83-5848-4b0c-9d49-6681582c6ede\") "
Nov 04 03:25:59.079704 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:59.079523    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/47389e83-5848-4b0c-9d49-6681582c6ede-webhook-cert\" (UniqueName: \"kubernetes.io/secret/47389e83-5848-4b0c-9d49-6681582c6ede-webhook-cert\") pod \"47389e83-5848-4b0c-9d49-6681582c6ede\" (UID: \"47389e83-5848-4b0c-9d49-6681582c6ede\") "
Nov 04 03:25:59.079704 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:59.079653    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/47389e83-5848-4b0c-9d49-6681582c6ede-metrics-certs\" (UniqueName: \"kubernetes.io/secret/47389e83-5848-4b0c-9d49-6681582c6ede-metrics-certs\") pod \"47389e83-5848-4b0c-9d49-6681582c6ede\" (UID: \"47389e83-5848-4b0c-9d49-6681582c6ede\") "
Nov 04 03:25:59.082189 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:59.082155    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47389e83-5848-4b0c-9d49-6681582c6ede-metrics-certs" pod "47389e83-5848-4b0c-9d49-6681582c6ede" (UID: "47389e83-5848-4b0c-9d49-6681582c6ede"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:25:59.082571 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:59.082544    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47389e83-5848-4b0c-9d49-6681582c6ede-kube-api-access-clmqx" pod "47389e83-5848-4b0c-9d49-6681582c6ede" (UID: "47389e83-5848-4b0c-9d49-6681582c6ede"). InnerVolumeSpecName "kube-api-access-clmqx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:25:59.084537 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:59.084498    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47389e83-5848-4b0c-9d49-6681582c6ede-webhook-cert" pod "47389e83-5848-4b0c-9d49-6681582c6ede" (UID: "47389e83-5848-4b0c-9d49-6681582c6ede"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:25:59.180977 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:59.180900    5609 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47389e83-5848-4b0c-9d49-6681582c6ede-metrics-certs\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:25:59.180977 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:59.180935    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-clmqx\" (UniqueName: \"kubernetes.io/projected/47389e83-5848-4b0c-9d49-6681582c6ede-kube-api-access-clmqx\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:25:59.180977 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:59.180949    5609 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/47389e83-5848-4b0c-9d49-6681582c6ede-webhook-cert\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:25:59.503007 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:59.502970    5609 generic.go:356] "Generic (PLEG): container finished" podID="47389e83-5848-4b0c-9d49-6681582c6ede" containerID="4337ffbdecae153d64dd8a49fa99528b5d63cd6120190ec0e10175de569f9ea1" exitCode=0
Nov 04 03:25:59.503435 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:59.503013    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-589f8697fb-5r6hk"
Nov 04 03:25:59.503435 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:59.503057    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-589f8697fb-5r6hk" event={"ID":"47389e83-5848-4b0c-9d49-6681582c6ede","Type":"ContainerDied","Data":"4337ffbdecae153d64dd8a49fa99528b5d63cd6120190ec0e10175de569f9ea1"}
Nov 04 03:25:59.503435 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:59.503116    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-589f8697fb-5r6hk" event={"ID":"47389e83-5848-4b0c-9d49-6681582c6ede","Type":"ContainerDied","Data":"79bb0f4c39380fab5d531d13d6ea8aba271bdf58c594761d24cb6d76aa56a608"}
Nov 04 03:25:59.503435 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:59.503139    5609 scope.go:122] "RemoveContainer" containerID="4337ffbdecae153d64dd8a49fa99528b5d63cd6120190ec0e10175de569f9ea1"
Nov 04 03:25:59.516947 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:59.516926    5609 scope.go:122] "RemoveContainer" containerID="4337ffbdecae153d64dd8a49fa99528b5d63cd6120190ec0e10175de569f9ea1"
Nov 04 03:25:59.517243 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:25:59.517219    5609 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4337ffbdecae153d64dd8a49fa99528b5d63cd6120190ec0e10175de569f9ea1\": container with ID starting with 4337ffbdecae153d64dd8a49fa99528b5d63cd6120190ec0e10175de569f9ea1 not found: ID does not exist" containerID="4337ffbdecae153d64dd8a49fa99528b5d63cd6120190ec0e10175de569f9ea1"
Nov 04 03:25:59.517344 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:59.517255    5609 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4337ffbdecae153d64dd8a49fa99528b5d63cd6120190ec0e10175de569f9ea1"} err="failed to get container status \"4337ffbdecae153d64dd8a49fa99528b5d63cd6120190ec0e10175de569f9ea1\": rpc error: code = NotFound desc = could not find container \"4337ffbdecae153d64dd8a49fa99528b5d63cd6120190ec0e10175de569f9ea1\": container with ID starting with 4337ffbdecae153d64dd8a49fa99528b5d63cd6120190ec0e10175de569f9ea1 not found: ID does not exist"
Nov 04 03:25:59.660149 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:59.660107    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-589f8697fb-5r6hk"]
Nov 04 03:25:59.707124 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:25:59.707084    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-589f8697fb-5r6hk"]
Nov 04 03:26:01.535653 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:01.535620    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="47389e83-5848-4b0c-9d49-6681582c6ede" path="/var/lib/kubelet/pods/47389e83-5848-4b0c-9d49-6681582c6ede/volumes"
Nov 04 03:26:03.894667 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:03.894631    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-pmscg"]
Nov 04 03:26:03.895189 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:03.895167    5609 cpu_manager.go:414] "containerMap: removing container" podUID="47389e83-5848-4b0c-9d49-6681582c6ede" containerName="manager"
Nov 04 03:26:03.895274 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:03.895192    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="47389e83-5848-4b0c-9d49-6681582c6ede" containerName="manager"
Nov 04 03:26:03.895326 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:03.895301    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="47389e83-5848-4b0c-9d49-6681582c6ede" containerName="manager"
Nov 04 03:26:03.898414 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:03.898380    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-pmscg"
Nov 04 03:26:03.953311 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:03.953282    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Nov 04 03:26:03.953458 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:03.953282    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Nov 04 03:26:03.999986 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:03.999961    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Nov 04 03:26:04.000337 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:04.000317    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-868th\""
Nov 04 03:26:04.002621 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:04.002600    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-pmscg"]
Nov 04 03:26:04.021745 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:04.021721    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/22e952c0-bbe0-4c13-ab52-31e9a96aab29-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-pmscg\" (UID: \"22e952c0-bbe0-4c13-ab52-31e9a96aab29\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-pmscg"
Nov 04 03:26:04.021844 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:04.021754    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/22e952c0-bbe0-4c13-ab52-31e9a96aab29-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-pmscg\" (UID: \"22e952c0-bbe0-4c13-ab52-31e9a96aab29\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-pmscg"
Nov 04 03:26:04.021844 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:04.021774    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/22e952c0-bbe0-4c13-ab52-31e9a96aab29-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-pmscg\" (UID: \"22e952c0-bbe0-4c13-ab52-31e9a96aab29\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-pmscg"
Nov 04 03:26:04.021918 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:04.021851    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/22e952c0-bbe0-4c13-ab52-31e9a96aab29-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-pmscg\" (UID: \"22e952c0-bbe0-4c13-ab52-31e9a96aab29\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-pmscg"
Nov 04 03:26:04.021918 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:04.021907    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22e952c0-bbe0-4c13-ab52-31e9a96aab29-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-pmscg\" (UID: \"22e952c0-bbe0-4c13-ab52-31e9a96aab29\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-pmscg"
Nov 04 03:26:04.021988 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:04.021976    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/22e952c0-bbe0-4c13-ab52-31e9a96aab29-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-pmscg\" (UID: \"22e952c0-bbe0-4c13-ab52-31e9a96aab29\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-pmscg"
Nov 04 03:26:04.022035 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:04.021995    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwx5n\" (UniqueName: \"kubernetes.io/projected/22e952c0-bbe0-4c13-ab52-31e9a96aab29-kube-api-access-cwx5n\") pod \"istiod-openshift-gateway-7ddd9cfdc8-pmscg\" (UID: \"22e952c0-bbe0-4c13-ab52-31e9a96aab29\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-pmscg"
Nov 04 03:26:04.123294 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:04.123257    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/22e952c0-bbe0-4c13-ab52-31e9a96aab29-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-pmscg\" (UID: \"22e952c0-bbe0-4c13-ab52-31e9a96aab29\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-pmscg"
Nov 04 03:26:04.123294 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:04.123299    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22e952c0-bbe0-4c13-ab52-31e9a96aab29-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-pmscg\" (UID: \"22e952c0-bbe0-4c13-ab52-31e9a96aab29\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-pmscg"
Nov 04 03:26:04.123538 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:04.123343    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/22e952c0-bbe0-4c13-ab52-31e9a96aab29-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-pmscg\" (UID: \"22e952c0-bbe0-4c13-ab52-31e9a96aab29\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-pmscg"
Nov 04 03:26:04.123538 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:04.123360    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwx5n\" (UniqueName: \"kubernetes.io/projected/22e952c0-bbe0-4c13-ab52-31e9a96aab29-kube-api-access-cwx5n\") pod \"istiod-openshift-gateway-7ddd9cfdc8-pmscg\" (UID: \"22e952c0-bbe0-4c13-ab52-31e9a96aab29\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-pmscg"
Nov 04 03:26:04.123538 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:04.123431    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/22e952c0-bbe0-4c13-ab52-31e9a96aab29-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-pmscg\" (UID: \"22e952c0-bbe0-4c13-ab52-31e9a96aab29\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-pmscg"
Nov 04 03:26:04.123538 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:04.123460    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/22e952c0-bbe0-4c13-ab52-31e9a96aab29-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-pmscg\" (UID: \"22e952c0-bbe0-4c13-ab52-31e9a96aab29\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-pmscg"
Nov 04 03:26:04.123538 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:04.123490    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/22e952c0-bbe0-4c13-ab52-31e9a96aab29-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-pmscg\" (UID: \"22e952c0-bbe0-4c13-ab52-31e9a96aab29\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-pmscg"
Nov 04 03:26:04.124257 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:04.124229    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/22e952c0-bbe0-4c13-ab52-31e9a96aab29-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-pmscg\" (UID: \"22e952c0-bbe0-4c13-ab52-31e9a96aab29\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-pmscg"
Nov 04 03:26:04.125610 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:04.125585    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/22e952c0-bbe0-4c13-ab52-31e9a96aab29-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-pmscg\" (UID: \"22e952c0-bbe0-4c13-ab52-31e9a96aab29\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-pmscg"
Nov 04 03:26:04.125727 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:04.125708    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/22e952c0-bbe0-4c13-ab52-31e9a96aab29-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-pmscg\" (UID: \"22e952c0-bbe0-4c13-ab52-31e9a96aab29\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-pmscg"
Nov 04 03:26:04.125855 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:04.125835    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/22e952c0-bbe0-4c13-ab52-31e9a96aab29-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-pmscg\" (UID: \"22e952c0-bbe0-4c13-ab52-31e9a96aab29\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-pmscg"
Nov 04 03:26:04.125891 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:04.125866    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/22e952c0-bbe0-4c13-ab52-31e9a96aab29-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-pmscg\" (UID: \"22e952c0-bbe0-4c13-ab52-31e9a96aab29\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-pmscg"
Nov 04 03:26:04.182810 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:04.182748    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22e952c0-bbe0-4c13-ab52-31e9a96aab29-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-pmscg\" (UID: \"22e952c0-bbe0-4c13-ab52-31e9a96aab29\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-pmscg"
Nov 04 03:26:04.182925 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:04.182859    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwx5n\" (UniqueName: \"kubernetes.io/projected/22e952c0-bbe0-4c13-ab52-31e9a96aab29-kube-api-access-cwx5n\") pod \"istiod-openshift-gateway-7ddd9cfdc8-pmscg\" (UID: \"22e952c0-bbe0-4c13-ab52-31e9a96aab29\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-pmscg"
Nov 04 03:26:04.209872 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:04.209842    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-pmscg"
Nov 04 03:26:04.541220 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:04.541136    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-98hr6" event={"ID":"c8183d45-a9c7-4fe4-bbfa-a1ddc7e33423","Type":"ContainerStarted","Data":"31c5b280894c4bcbf09344a9b7ad981deb62509dae4f18ca9e1379fa49027f86"}
Nov 04 03:26:04.541363 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:04.541226    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-98hr6"
Nov 04 03:26:04.648725 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:04.648693    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-pmscg"]
Nov 04 03:26:04.653715 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:26:04.653686    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22e952c0_bbe0_4c13_ab52_31e9a96aab29.slice/crio-941a074a9e17da960bb6a51280a323061bcd8b76bf3d121ecbf048522f8809ed WatchSource:0}: Error finding container 941a074a9e17da960bb6a51280a323061bcd8b76bf3d121ecbf048522f8809ed: Status 404 returned error can't find the container with id 941a074a9e17da960bb6a51280a323061bcd8b76bf3d121ecbf048522f8809ed
Nov 04 03:26:04.654704 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:04.654667    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-98hr6" podStartSLOduration=2.985154402 podStartE2EDuration="15.654654258s" totalImagesPullingTime="12.669499856s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:25:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:26:04.653063386 +0000 UTC m=+507.229716302" watchObservedRunningTime="2026-11-04 03:26:04.654654258 +0000 UTC m=+507.231307171"
Nov 04 03:26:05.547272 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:05.547234    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-pmscg" event={"ID":"22e952c0-bbe0-4c13-ab52-31e9a96aab29","Type":"ContainerStarted","Data":"941a074a9e17da960bb6a51280a323061bcd8b76bf3d121ecbf048522f8809ed"}
Nov 04 03:26:08.773334 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:08.773298    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:26:08.773700 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:08.773368    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:26:09.572150 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:09.572114    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-pmscg" event={"ID":"22e952c0-bbe0-4c13-ab52-31e9a96aab29","Type":"ContainerStarted","Data":"ddd3543d413a969d002f0107f388a7d76d08876ddf5442507a9aed47f4a2d188"}
Nov 04 03:26:09.572352 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:09.572311    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-pmscg"
Nov 04 03:26:09.574066 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:09.574036    5609 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-pmscg container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 03:26:09.574195 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:09.574090    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-pmscg" podUID="22e952c0-bbe0-4c13-ab52-31e9a96aab29" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 03:26:10.579033 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:10.579006    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-pmscg"
Nov 04 03:26:10.695412 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:10.695349    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-pmscg" podStartSLOduration=3.578630876 podStartE2EDuration="7.695335281s" totalImagesPullingTime="4.116704405s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:26:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:26:09.769585933 +0000 UTC m=+512.346238848" watchObservedRunningTime="2026-11-04 03:26:10.695335281 +0000 UTC m=+513.271988196"
Nov 04 03:26:15.553411 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:15.553360    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-98hr6"
Nov 04 03:26:18.559838 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:26:18.559802    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:26:27.948617 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:27.948571    5609 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-v5h77 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 03:26:27.949068 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:27.948657    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77" podUID="8772b2ff-e4cc-4b41-85a9-491fbaf001c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 03:26:30.693093 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:30.693059    5609 generic.go:356] "Generic (PLEG): container finished" podID="8772b2ff-e4cc-4b41-85a9-491fbaf001c8" containerID="ce2fe624e38171c11e491aa38095b48ba3c41cfd3675b824bef1436e712f344e" exitCode=0
Nov 04 03:26:30.693518 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:30.693132    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77" event={"ID":"8772b2ff-e4cc-4b41-85a9-491fbaf001c8","Type":"ContainerDied","Data":"ce2fe624e38171c11e491aa38095b48ba3c41cfd3675b824bef1436e712f344e"}
Nov 04 03:26:30.693518 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:30.693490    5609 scope.go:122] "RemoveContainer" containerID="ce2fe624e38171c11e491aa38095b48ba3c41cfd3675b824bef1436e712f344e"
Nov 04 03:26:30.694666 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:30.694632    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:26:30.694739 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:30.694707    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:26:30.694775 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:30.694760    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:26:31.713700 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:31.713662    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77" event={"ID":"8772b2ff-e4cc-4b41-85a9-491fbaf001c8","Type":"ContainerStarted","Data":"25f5f576299df829ea29b5a2884c925403eae441dee66f2dc64b0eb860c69393"}
Nov 04 03:26:31.906422 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:31.905781    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77"
Nov 04 03:26:31.906422 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:31.905907    5609 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-v5h77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body=
Nov 04 03:26:31.906422 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:31.905965    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77" podUID="8772b2ff-e4cc-4b41-85a9-491fbaf001c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused"
Nov 04 03:26:32.736880 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:32.736846    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6f7c4f5d87-28j47"]
Nov 04 03:26:32.739907 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:32.739885    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f7c4f5d87-28j47"
Nov 04 03:26:32.809117 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:32.809014    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6f7c4f5d87-28j47"]
Nov 04 03:26:32.818275 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:32.818245    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/667a921d-333d-4244-8ea8-f0a60692a5f8-tls-cert\") pod \"authorino-6f7c4f5d87-28j47\" (UID: \"667a921d-333d-4244-8ea8-f0a60692a5f8\") " pod="kuadrant-system/authorino-6f7c4f5d87-28j47"
Nov 04 03:26:32.818558 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:32.818536    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8mvd\" (UniqueName: \"kubernetes.io/projected/667a921d-333d-4244-8ea8-f0a60692a5f8-kube-api-access-h8mvd\") pod \"authorino-6f7c4f5d87-28j47\" (UID: \"667a921d-333d-4244-8ea8-f0a60692a5f8\") " pod="kuadrant-system/authorino-6f7c4f5d87-28j47"
Nov 04 03:26:32.843868 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:32.843841    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Nov 04 03:26:32.843868 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:32.843857    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-trsqp\""
Nov 04 03:26:32.908428 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:32.904760    5609 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-v5h77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body=
Nov 04 03:26:32.908428 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:32.904850    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77" podUID="8772b2ff-e4cc-4b41-85a9-491fbaf001c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused"
Nov 04 03:26:32.919020 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:32.918990    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8mvd\" (UniqueName: \"kubernetes.io/projected/667a921d-333d-4244-8ea8-f0a60692a5f8-kube-api-access-h8mvd\") pod \"authorino-6f7c4f5d87-28j47\" (UID: \"667a921d-333d-4244-8ea8-f0a60692a5f8\") " pod="kuadrant-system/authorino-6f7c4f5d87-28j47"
Nov 04 03:26:32.919157 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:32.919104    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/667a921d-333d-4244-8ea8-f0a60692a5f8-tls-cert\") pod \"authorino-6f7c4f5d87-28j47\" (UID: \"667a921d-333d-4244-8ea8-f0a60692a5f8\") " pod="kuadrant-system/authorino-6f7c4f5d87-28j47"
Nov 04 03:26:32.921607 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:32.921575    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/667a921d-333d-4244-8ea8-f0a60692a5f8-tls-cert\") pod \"authorino-6f7c4f5d87-28j47\" (UID: \"667a921d-333d-4244-8ea8-f0a60692a5f8\") " pod="kuadrant-system/authorino-6f7c4f5d87-28j47"
Nov 04 03:26:32.977011 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:32.976987    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8mvd\" (UniqueName: \"kubernetes.io/projected/667a921d-333d-4244-8ea8-f0a60692a5f8-kube-api-access-h8mvd\") pod \"authorino-6f7c4f5d87-28j47\" (UID: \"667a921d-333d-4244-8ea8-f0a60692a5f8\") " pod="kuadrant-system/authorino-6f7c4f5d87-28j47"
Nov 04 03:26:33.051297 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:33.051216    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f7c4f5d87-28j47"
Nov 04 03:26:33.443043 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:33.443010    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6f7c4f5d87-28j47"]
Nov 04 03:26:33.448084 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:26:33.448054    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod667a921d_333d_4244_8ea8_f0a60692a5f8.slice/crio-e18f8066b0008891f2b942c39fdd4f024720015205c4e7e2ab9bdf667fda8df5 WatchSource:0}: Error finding container e18f8066b0008891f2b942c39fdd4f024720015205c4e7e2ab9bdf667fda8df5: Status 404 returned error can't find the container with id e18f8066b0008891f2b942c39fdd4f024720015205c4e7e2ab9bdf667fda8df5
Nov 04 03:26:33.725920 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:33.725836    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f7c4f5d87-28j47" event={"ID":"667a921d-333d-4244-8ea8-f0a60692a5f8","Type":"ContainerStarted","Data":"e18f8066b0008891f2b942c39fdd4f024720015205c4e7e2ab9bdf667fda8df5"}
Nov 04 03:26:33.903426 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:33.897789    5609 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-v5h77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body=
Nov 04 03:26:33.903426 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:33.897850    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77" podUID="8772b2ff-e4cc-4b41-85a9-491fbaf001c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused"
Nov 04 03:26:34.901872 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:34.901838    5609 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-v5h77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body=
Nov 04 03:26:34.902128 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:34.902088    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77" podUID="8772b2ff-e4cc-4b41-85a9-491fbaf001c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused"
Nov 04 03:26:35.747147 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:35.747111    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f7c4f5d87-28j47" event={"ID":"667a921d-333d-4244-8ea8-f0a60692a5f8","Type":"ContainerStarted","Data":"0009f4e4923991151afda5126bc1b9bf387fc1ffce4bed14b527bd48f3fd7829"}
Nov 04 03:26:35.867663 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:35.867610    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-6f7c4f5d87-28j47" podStartSLOduration=2.445739632 podStartE2EDuration="3.867596106s" totalImagesPullingTime="1.421856474s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:26:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:26:35.866243843 +0000 UTC m=+538.442896757" watchObservedRunningTime="2026-11-04 03:26:35.867596106 +0000 UTC m=+538.444249019"
Nov 04 03:26:35.900423 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:35.898814    5609 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-v5h77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body=
Nov 04 03:26:35.900423 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:35.898893    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77" podUID="8772b2ff-e4cc-4b41-85a9-491fbaf001c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused"
Nov 04 03:26:36.901555 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:36.901524    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77"
Nov 04 03:26:36.902036 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:36.901902    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77"
Nov 04 03:26:36.902630 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:26:36.902612    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-v5h77"
Nov 04 03:26:48.562585 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:26:48.562502    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:27:18.558704 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:27:18.558667    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:27:37.584628 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:27:37.584588    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-p9989_068619f8-7fbb-4b22-ac7d-a1f144e084c0/console-operator/2.log"
Nov 04 03:27:37.589112 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:27:37.589087    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-p9989_068619f8-7fbb-4b22-ac7d-a1f144e084c0/console-operator/2.log"
Nov 04 03:27:37.591619 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:27:37.591601    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vmw5z_21a42913-5c4e-4057-aae5-2804038f16e4/ovn-acl-logging/0.log"
Nov 04 03:27:37.596336 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:27:37.596318    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vmw5z_21a42913-5c4e-4057-aae5-2804038f16e4/ovn-acl-logging/0.log"
Nov 04 03:27:48.559644 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:27:48.559600    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:28:13.300017 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:28:13.299984    5609 generic.go:356] "Generic (PLEG): container finished" podID="e6af8c02-22bc-49e1-adb9-2f01da0af47c" containerID="509dcec1e5785ea67375bc0593810695134495c9bc5c13bae31f47e52dc68238" exitCode=1
Nov 04 03:28:13.300569 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:28:13.300064    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5fb896bc89-fkk4g" event={"ID":"e6af8c02-22bc-49e1-adb9-2f01da0af47c","Type":"ContainerDied","Data":"509dcec1e5785ea67375bc0593810695134495c9bc5c13bae31f47e52dc68238"}
Nov 04 03:28:13.300569 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:28:13.300506    5609 scope.go:122] "RemoveContainer" containerID="509dcec1e5785ea67375bc0593810695134495c9bc5c13bae31f47e52dc68238"
Nov 04 03:28:13.300902 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:28:13.300885    5609 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 03:28:15.313183 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:28:15.313153    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5fb896bc89-fkk4g" event={"ID":"e6af8c02-22bc-49e1-adb9-2f01da0af47c","Type":"ContainerStarted","Data":"456b2f38a64c0d3e9fdb591109284af6e54d2ca57935a2afa20e72aa515a05e6"}
Nov 04 03:28:15.313598 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:28:15.313346    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5fb896bc89-fkk4g"
Nov 04 03:28:18.559665 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:28:18.559580    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:28:26.324126 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:28:26.324092    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5fb896bc89-fkk4g"
Nov 04 03:28:48.559332 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:28:48.559295    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:28:56.815494 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:28:56.815439    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-tbfbx"]
Nov 04 03:28:56.819396 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:28:56.819366    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-tbfbx"
Nov 04 03:28:56.869189 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:28:56.869159    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Nov 04 03:28:56.915997 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:28:56.915953    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Nov 04 03:28:56.916255 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:28:56.915998    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-66tnw\""
Nov 04 03:28:56.916255 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:28:56.916242    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"echo-simulator-kserve-self-signed-certs\""
Nov 04 03:28:56.918527 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:28:56.918500    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-tbfbx"]
Nov 04 03:28:56.966530 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:28:56.964856    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3ab7fb6-0416-4f11-85f4-555ffc5e0c56-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-tbfbx\" (UID: \"a3ab7fb6-0416-4f11-85f4-555ffc5e0c56\") " pod="llm/echo-simulator-kserve-64b6f8f96c-tbfbx"
Nov 04 03:28:56.966530 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:28:56.965052    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3ab7fb6-0416-4f11-85f4-555ffc5e0c56-home\") pod \"echo-simulator-kserve-64b6f8f96c-tbfbx\" (UID: \"a3ab7fb6-0416-4f11-85f4-555ffc5e0c56\") " pod="llm/echo-simulator-kserve-64b6f8f96c-tbfbx"
Nov 04 03:28:56.966530 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:28:56.965098    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3ab7fb6-0416-4f11-85f4-555ffc5e0c56-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-tbfbx\" (UID: \"a3ab7fb6-0416-4f11-85f4-555ffc5e0c56\") " pod="llm/echo-simulator-kserve-64b6f8f96c-tbfbx"
Nov 04 03:28:56.966530 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:28:56.965132    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3ab7fb6-0416-4f11-85f4-555ffc5e0c56-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-tbfbx\" (UID: \"a3ab7fb6-0416-4f11-85f4-555ffc5e0c56\") " pod="llm/echo-simulator-kserve-64b6f8f96c-tbfbx"
Nov 04 03:28:56.966530 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:28:56.965185    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6sm7\" (UniqueName: \"kubernetes.io/projected/a3ab7fb6-0416-4f11-85f4-555ffc5e0c56-kube-api-access-k6sm7\") pod \"echo-simulator-kserve-64b6f8f96c-tbfbx\" (UID: \"a3ab7fb6-0416-4f11-85f4-555ffc5e0c56\") " pod="llm/echo-simulator-kserve-64b6f8f96c-tbfbx"
Nov 04 03:28:56.966530 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:28:56.965220    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3ab7fb6-0416-4f11-85f4-555ffc5e0c56-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-tbfbx\" (UID: \"a3ab7fb6-0416-4f11-85f4-555ffc5e0c56\") " pod="llm/echo-simulator-kserve-64b6f8f96c-tbfbx"
Nov 04 03:28:57.066088 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:28:57.066007    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3ab7fb6-0416-4f11-85f4-555ffc5e0c56-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-tbfbx\" (UID: \"a3ab7fb6-0416-4f11-85f4-555ffc5e0c56\") " pod="llm/echo-simulator-kserve-64b6f8f96c-tbfbx"
Nov 04 03:28:57.066088 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:28:57.066085    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3ab7fb6-0416-4f11-85f4-555ffc5e0c56-home\") pod \"echo-simulator-kserve-64b6f8f96c-tbfbx\" (UID: \"a3ab7fb6-0416-4f11-85f4-555ffc5e0c56\") " pod="llm/echo-simulator-kserve-64b6f8f96c-tbfbx"
Nov 04 03:28:57.066264 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:28:57.066107    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3ab7fb6-0416-4f11-85f4-555ffc5e0c56-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-tbfbx\" (UID: \"a3ab7fb6-0416-4f11-85f4-555ffc5e0c56\") " pod="llm/echo-simulator-kserve-64b6f8f96c-tbfbx"
Nov 04 03:28:57.066264 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:28:57.066129    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3ab7fb6-0416-4f11-85f4-555ffc5e0c56-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-tbfbx\" (UID: \"a3ab7fb6-0416-4f11-85f4-555ffc5e0c56\") " pod="llm/echo-simulator-kserve-64b6f8f96c-tbfbx"
Nov 04 03:28:57.066264 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:28:57.066176    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6sm7\" (UniqueName: \"kubernetes.io/projected/a3ab7fb6-0416-4f11-85f4-555ffc5e0c56-kube-api-access-k6sm7\") pod \"echo-simulator-kserve-64b6f8f96c-tbfbx\" (UID: \"a3ab7fb6-0416-4f11-85f4-555ffc5e0c56\") " pod="llm/echo-simulator-kserve-64b6f8f96c-tbfbx"
Nov 04 03:28:57.066264 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:28:57.066208    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3ab7fb6-0416-4f11-85f4-555ffc5e0c56-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-tbfbx\" (UID: \"a3ab7fb6-0416-4f11-85f4-555ffc5e0c56\") " pod="llm/echo-simulator-kserve-64b6f8f96c-tbfbx"
Nov 04 03:28:57.066506 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:28:57.066483    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3ab7fb6-0416-4f11-85f4-555ffc5e0c56-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-tbfbx\" (UID: \"a3ab7fb6-0416-4f11-85f4-555ffc5e0c56\") " pod="llm/echo-simulator-kserve-64b6f8f96c-tbfbx"
Nov 04 03:28:57.066553 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:28:57.066500    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3ab7fb6-0416-4f11-85f4-555ffc5e0c56-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-tbfbx\" (UID: \"a3ab7fb6-0416-4f11-85f4-555ffc5e0c56\") " pod="llm/echo-simulator-kserve-64b6f8f96c-tbfbx"
Nov 04 03:28:57.066602 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:28:57.066559    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3ab7fb6-0416-4f11-85f4-555ffc5e0c56-home\") pod \"echo-simulator-kserve-64b6f8f96c-tbfbx\" (UID: \"a3ab7fb6-0416-4f11-85f4-555ffc5e0c56\") " pod="llm/echo-simulator-kserve-64b6f8f96c-tbfbx"
Nov 04 03:28:57.068687 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:28:57.068664    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3ab7fb6-0416-4f11-85f4-555ffc5e0c56-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-tbfbx\" (UID: \"a3ab7fb6-0416-4f11-85f4-555ffc5e0c56\") " pod="llm/echo-simulator-kserve-64b6f8f96c-tbfbx"
Nov 04 03:28:57.069074 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:28:57.069052    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3ab7fb6-0416-4f11-85f4-555ffc5e0c56-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-tbfbx\" (UID: \"a3ab7fb6-0416-4f11-85f4-555ffc5e0c56\") " pod="llm/echo-simulator-kserve-64b6f8f96c-tbfbx"
Nov 04 03:28:57.126484 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:28:57.126448    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6sm7\" (UniqueName: \"kubernetes.io/projected/a3ab7fb6-0416-4f11-85f4-555ffc5e0c56-kube-api-access-k6sm7\") pod \"echo-simulator-kserve-64b6f8f96c-tbfbx\" (UID: \"a3ab7fb6-0416-4f11-85f4-555ffc5e0c56\") " pod="llm/echo-simulator-kserve-64b6f8f96c-tbfbx"
Nov 04 03:28:57.137209 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:28:57.137165    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-tbfbx"
Nov 04 03:28:57.523437 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:28:57.523384    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-tbfbx"]
Nov 04 03:28:57.528849 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:28:57.528808    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3ab7fb6_0416_4f11_85f4_555ffc5e0c56.slice/crio-8873c3b35465f1ac364dd087ff1a0e4b289496c6e43446139fd54ee52cb4d07c WatchSource:0}: Error finding container 8873c3b35465f1ac364dd087ff1a0e4b289496c6e43446139fd54ee52cb4d07c: Status 404 returned error can't find the container with id 8873c3b35465f1ac364dd087ff1a0e4b289496c6e43446139fd54ee52cb4d07c
Nov 04 03:28:57.564756 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:28:57.564723    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-tbfbx" event={"ID":"a3ab7fb6-0416-4f11-85f4-555ffc5e0c56","Type":"ContainerStarted","Data":"8873c3b35465f1ac364dd087ff1a0e4b289496c6e43446139fd54ee52cb4d07c"}
Nov 04 03:29:00.584072 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:00.584036    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-tbfbx" event={"ID":"a3ab7fb6-0416-4f11-85f4-555ffc5e0c56","Type":"ContainerStarted","Data":"acd960715ecbf7f0e4479fa13525d6c145e653e3cddb8577b188da409ab4d986"}
Nov 04 03:29:00.698364 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:00.698313    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/echo-simulator-kserve-64b6f8f96c-tbfbx" podStartSLOduration=2.30060948 podStartE2EDuration="4.698300478s" totalImagesPullingTime="2.397690998s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:28:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:29:00.696079845 +0000 UTC m=+683.272732769" watchObservedRunningTime="2026-11-04 03:29:00.698300478 +0000 UTC m=+683.274953391"
Nov 04 03:29:01.137254 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:01.137218    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/echo-simulator-kserve-64b6f8f96c-tbfbx"
Nov 04 03:29:07.137343 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:07.137303    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/echo-simulator-kserve-64b6f8f96c-tbfbx"
Nov 04 03:29:07.155188 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:07.155160    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/echo-simulator-kserve-64b6f8f96c-tbfbx"
Nov 04 03:29:07.640782 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:07.640749    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/echo-simulator-kserve-64b6f8f96c-tbfbx"
Nov 04 03:29:17.483764 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:17.483731    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd"]
Nov 04 03:29:17.487883 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:17.487865    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd"
Nov 04 03:29:17.563173 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:17.563139    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-2tbrd\" (UID: \"fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd"
Nov 04 03:29:17.563173 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:17.563173    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7fvv\" (UniqueName: \"kubernetes.io/projected/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-kube-api-access-q7fvv\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-2tbrd\" (UID: \"fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd"
Nov 04 03:29:17.563375 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:17.563193    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-2tbrd\" (UID: \"fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd"
Nov 04 03:29:17.563375 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:17.563264    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-2tbrd\" (UID: \"fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd"
Nov 04 03:29:17.563375 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:17.563331    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-2tbrd\" (UID: \"fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd"
Nov 04 03:29:17.563375 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:17.563373    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-2tbrd\" (UID: \"fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd"
Nov 04 03:29:17.563536 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:17.563495    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-2tbrd\" (UID: \"fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd"
Nov 04 03:29:17.563536 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:17.563511    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-2tbrd\" (UID: \"fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd"
Nov 04 03:29:17.563536 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:17.563526    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-2tbrd\" (UID: \"fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd"
Nov 04 03:29:17.587209 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:17.587182    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd"]
Nov 04 03:29:17.664263 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:17.664221    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-2tbrd\" (UID: \"fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd"
Nov 04 03:29:17.664263 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:17.664264    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-2tbrd\" (UID: \"fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd"
Nov 04 03:29:17.664549 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:17.664301    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-2tbrd\" (UID: \"fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd"
Nov 04 03:29:17.664549 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:17.664390    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-2tbrd\" (UID: \"fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd"
Nov 04 03:29:17.664549 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:17.664437    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-2tbrd\" (UID: \"fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd"
Nov 04 03:29:17.664549 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:17.664456    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-2tbrd\" (UID: \"fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd"
Nov 04 03:29:17.664549 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:17.664502    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-2tbrd\" (UID: \"fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd"
Nov 04 03:29:17.664549 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:17.664527    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7fvv\" (UniqueName: \"kubernetes.io/projected/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-kube-api-access-q7fvv\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-2tbrd\" (UID: \"fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd"
Nov 04 03:29:17.664820 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:17.664552    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-2tbrd\" (UID: \"fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd"
Nov 04 03:29:17.664882 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:17.664807    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-2tbrd\" (UID: \"fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd"
Nov 04 03:29:17.664882 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:17.664862    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-2tbrd\" (UID: \"fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd"
Nov 04 03:29:17.665019 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:17.665000    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-2tbrd\" (UID: \"fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd"
Nov 04 03:29:17.665075 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:17.665045    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-2tbrd\" (UID: \"fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd"
Nov 04 03:29:17.665112 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:17.665073    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-2tbrd\" (UID: \"fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd"
Nov 04 03:29:17.666745 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:17.666723    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-2tbrd\" (UID: \"fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd"
Nov 04 03:29:17.666967 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:17.666949    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-2tbrd\" (UID: \"fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd"
Nov 04 03:29:17.725646 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:17.725612    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7fvv\" (UniqueName: \"kubernetes.io/projected/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-kube-api-access-q7fvv\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-2tbrd\" (UID: \"fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd"
Nov 04 03:29:17.730857 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:17.730829    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-2tbrd\" (UID: \"fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd"
Nov 04 03:29:17.801138 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:17.801055    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd"
Nov 04 03:29:18.228614 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:18.228584    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd"]
Nov 04 03:29:18.234059 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:29:18.234032    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd9f1f55_d4c4_42b4_8cc3_41a2f87a3a8a.slice/crio-efbebbb11fa0d5a099d1caffe18753900d9b828087208056e92884a629e0eeb8 WatchSource:0}: Error finding container efbebbb11fa0d5a099d1caffe18753900d9b828087208056e92884a629e0eeb8: Status 404 returned error can't find the container with id efbebbb11fa0d5a099d1caffe18753900d9b828087208056e92884a629e0eeb8
Nov 04 03:29:18.236158 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:18.236129    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:29:18.236227 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:18.236191    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:29:18.236227 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:18.236221    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:29:18.560185 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:29:18.560103    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:29:18.708166 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:18.708035    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd" event={"ID":"fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a","Type":"ContainerStarted","Data":"4baa554d96d0535db710ba2fd7ea5ac560841a0877a078c08e40bdfe5f5e6c5d"}
Nov 04 03:29:18.708166 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:18.708112    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd" event={"ID":"fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a","Type":"ContainerStarted","Data":"efbebbb11fa0d5a099d1caffe18753900d9b828087208056e92884a629e0eeb8"}
Nov 04 03:29:18.801667 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:18.801574    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd"
Nov 04 03:29:18.824174 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:18.824069    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd" podStartSLOduration=1.8240559410000001 podStartE2EDuration="1.824055941s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:29:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:29:18.821939697 +0000 UTC m=+701.398592645" watchObservedRunningTime="2026-11-04 03:29:18.824055941 +0000 UTC m=+701.400708854"
Nov 04 03:29:19.808599 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:19.806754    5609 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Nov 04 03:29:19.808599 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:19.806805    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd" podUID="fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Nov 04 03:29:20.805805 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:20.805773    5609 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Nov 04 03:29:20.806091 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:20.806064    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd" podUID="fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Nov 04 03:29:21.807839 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:21.807798    5609 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Nov 04 03:29:21.808450 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:21.808385    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd" podUID="fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Nov 04 03:29:22.808982 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:22.808935    5609 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Nov 04 03:29:22.809528 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:22.808998    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd" podUID="fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Nov 04 03:29:23.801796 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:23.801760    5609 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Nov 04 03:29:23.802083 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:23.802054    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd" podUID="fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Nov 04 03:29:24.801355 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:24.801325    5609 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Nov 04 03:29:24.801789 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:24.801376    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd" podUID="fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Nov 04 03:29:26.510789 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:26.510758    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd"
Nov 04 03:29:26.511255 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:26.511210    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd"
Nov 04 03:29:26.512141 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:26.512046    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd"
Nov 04 03:29:36.740671 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:36.740638    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5cb7d797b8-rjd4x"]
Nov 04 03:29:36.748962 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:36.748940    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5cb7d797b8-rjd4x"
Nov 04 03:29:36.799566 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:36.799536    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5cb7d797b8-rjd4x"]
Nov 04 03:29:36.866945 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:36.866914    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smk52\" (UniqueName: \"kubernetes.io/projected/b09e35e7-3a7f-4350-a3ef-61ba60f041a2-kube-api-access-smk52\") pod \"authorino-5cb7d797b8-rjd4x\" (UID: \"b09e35e7-3a7f-4350-a3ef-61ba60f041a2\") " pod="kuadrant-system/authorino-5cb7d797b8-rjd4x"
Nov 04 03:29:36.866945 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:36.866951    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b09e35e7-3a7f-4350-a3ef-61ba60f041a2-tls-cert\") pod \"authorino-5cb7d797b8-rjd4x\" (UID: \"b09e35e7-3a7f-4350-a3ef-61ba60f041a2\") " pod="kuadrant-system/authorino-5cb7d797b8-rjd4x"
Nov 04 03:29:36.968349 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:36.968315    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smk52\" (UniqueName: \"kubernetes.io/projected/b09e35e7-3a7f-4350-a3ef-61ba60f041a2-kube-api-access-smk52\") pod \"authorino-5cb7d797b8-rjd4x\" (UID: \"b09e35e7-3a7f-4350-a3ef-61ba60f041a2\") " pod="kuadrant-system/authorino-5cb7d797b8-rjd4x"
Nov 04 03:29:36.968530 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:36.968360    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b09e35e7-3a7f-4350-a3ef-61ba60f041a2-tls-cert\") pod \"authorino-5cb7d797b8-rjd4x\" (UID: \"b09e35e7-3a7f-4350-a3ef-61ba60f041a2\") " pod="kuadrant-system/authorino-5cb7d797b8-rjd4x"
Nov 04 03:29:36.970739 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:36.970719    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b09e35e7-3a7f-4350-a3ef-61ba60f041a2-tls-cert\") pod \"authorino-5cb7d797b8-rjd4x\" (UID: \"b09e35e7-3a7f-4350-a3ef-61ba60f041a2\") " pod="kuadrant-system/authorino-5cb7d797b8-rjd4x"
Nov 04 03:29:37.026423 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:37.026342    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-smk52\" (UniqueName: \"kubernetes.io/projected/b09e35e7-3a7f-4350-a3ef-61ba60f041a2-kube-api-access-smk52\") pod \"authorino-5cb7d797b8-rjd4x\" (UID: \"b09e35e7-3a7f-4350-a3ef-61ba60f041a2\") " pod="kuadrant-system/authorino-5cb7d797b8-rjd4x"
Nov 04 03:29:37.062121 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:37.062092    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5cb7d797b8-rjd4x"
Nov 04 03:29:37.434567 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:37.434532    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5cb7d797b8-rjd4x"]
Nov 04 03:29:37.439030 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:29:37.438999    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb09e35e7_3a7f_4350_a3ef_61ba60f041a2.slice/crio-5d4e1f35b857cd5376012d84ed9d1f53ed28813209a7bdf49df4d1f3f39d88b2 WatchSource:0}: Error finding container 5d4e1f35b857cd5376012d84ed9d1f53ed28813209a7bdf49df4d1f3f39d88b2: Status 404 returned error can't find the container with id 5d4e1f35b857cd5376012d84ed9d1f53ed28813209a7bdf49df4d1f3f39d88b2
Nov 04 03:29:37.821216 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:37.821132    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5cb7d797b8-rjd4x" event={"ID":"b09e35e7-3a7f-4350-a3ef-61ba60f041a2","Type":"ContainerStarted","Data":"5d4e1f35b857cd5376012d84ed9d1f53ed28813209a7bdf49df4d1f3f39d88b2"}
Nov 04 03:29:39.834377 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:39.834331    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5cb7d797b8-rjd4x" event={"ID":"b09e35e7-3a7f-4350-a3ef-61ba60f041a2","Type":"ContainerStarted","Data":"e1b05d3e9260648da2f222b4bc09aa7f6a0f6eb833e41178228ba1f6d30090b6"}
Nov 04 03:29:39.944989 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:39.944926    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-5cb7d797b8-rjd4x" podStartSLOduration=2.097980152 podStartE2EDuration="3.944910458s" totalImagesPullingTime="1.846930306s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:29:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:29:39.942256241 +0000 UTC m=+722.518909155" watchObservedRunningTime="2026-11-04 03:29:39.944910458 +0000 UTC m=+722.521563372"
Nov 04 03:29:39.993230 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:39.993193    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f7c4f5d87-28j47"]
Nov 04 03:29:39.993471 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:39.993443    5609 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-6f7c4f5d87-28j47" podUID="667a921d-333d-4244-8ea8-f0a60692a5f8" containerName="authorino" containerID="cri-o://0009f4e4923991151afda5126bc1b9bf387fc1ffce4bed14b527bd48f3fd7829" gracePeriod=30
Nov 04 03:29:40.255326 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:40.255294    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f7c4f5d87-28j47"
Nov 04 03:29:40.398964 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:40.398877    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/667a921d-333d-4244-8ea8-f0a60692a5f8-tls-cert\" (UniqueName: \"kubernetes.io/secret/667a921d-333d-4244-8ea8-f0a60692a5f8-tls-cert\") pod \"667a921d-333d-4244-8ea8-f0a60692a5f8\" (UID: \"667a921d-333d-4244-8ea8-f0a60692a5f8\") "
Nov 04 03:29:40.399116 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:40.399053    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/667a921d-333d-4244-8ea8-f0a60692a5f8-kube-api-access-h8mvd\" (UniqueName: \"kubernetes.io/projected/667a921d-333d-4244-8ea8-f0a60692a5f8-kube-api-access-h8mvd\") pod \"667a921d-333d-4244-8ea8-f0a60692a5f8\" (UID: \"667a921d-333d-4244-8ea8-f0a60692a5f8\") "
Nov 04 03:29:40.401219 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:40.401186    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/667a921d-333d-4244-8ea8-f0a60692a5f8-kube-api-access-h8mvd" pod "667a921d-333d-4244-8ea8-f0a60692a5f8" (UID: "667a921d-333d-4244-8ea8-f0a60692a5f8"). InnerVolumeSpecName "kube-api-access-h8mvd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:29:40.410554 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:40.410495    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/667a921d-333d-4244-8ea8-f0a60692a5f8-tls-cert" pod "667a921d-333d-4244-8ea8-f0a60692a5f8" (UID: "667a921d-333d-4244-8ea8-f0a60692a5f8"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:29:40.500502 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:40.500472    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h8mvd\" (UniqueName: \"kubernetes.io/projected/667a921d-333d-4244-8ea8-f0a60692a5f8-kube-api-access-h8mvd\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:29:40.500502 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:40.500499    5609 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/667a921d-333d-4244-8ea8-f0a60692a5f8-tls-cert\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:29:40.859684 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:40.859643    5609 generic.go:356] "Generic (PLEG): container finished" podID="667a921d-333d-4244-8ea8-f0a60692a5f8" containerID="0009f4e4923991151afda5126bc1b9bf387fc1ffce4bed14b527bd48f3fd7829" exitCode=0
Nov 04 03:29:40.860186 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:40.859700    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f7c4f5d87-28j47" event={"ID":"667a921d-333d-4244-8ea8-f0a60692a5f8","Type":"ContainerDied","Data":"0009f4e4923991151afda5126bc1b9bf387fc1ffce4bed14b527bd48f3fd7829"}
Nov 04 03:29:40.860186 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:40.859717    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f7c4f5d87-28j47"
Nov 04 03:29:40.860186 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:40.859750    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f7c4f5d87-28j47" event={"ID":"667a921d-333d-4244-8ea8-f0a60692a5f8","Type":"ContainerDied","Data":"e18f8066b0008891f2b942c39fdd4f024720015205c4e7e2ab9bdf667fda8df5"}
Nov 04 03:29:40.860186 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:40.859770    5609 scope.go:122] "RemoveContainer" containerID="0009f4e4923991151afda5126bc1b9bf387fc1ffce4bed14b527bd48f3fd7829"
Nov 04 03:29:40.877955 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:40.877930    5609 scope.go:122] "RemoveContainer" containerID="0009f4e4923991151afda5126bc1b9bf387fc1ffce4bed14b527bd48f3fd7829"
Nov 04 03:29:40.878256 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:29:40.878235    5609 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0009f4e4923991151afda5126bc1b9bf387fc1ffce4bed14b527bd48f3fd7829\": container with ID starting with 0009f4e4923991151afda5126bc1b9bf387fc1ffce4bed14b527bd48f3fd7829 not found: ID does not exist" containerID="0009f4e4923991151afda5126bc1b9bf387fc1ffce4bed14b527bd48f3fd7829"
Nov 04 03:29:40.878312 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:40.878265    5609 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0009f4e4923991151afda5126bc1b9bf387fc1ffce4bed14b527bd48f3fd7829"} err="failed to get container status \"0009f4e4923991151afda5126bc1b9bf387fc1ffce4bed14b527bd48f3fd7829\": rpc error: code = NotFound desc = could not find container \"0009f4e4923991151afda5126bc1b9bf387fc1ffce4bed14b527bd48f3fd7829\": container with ID starting with 0009f4e4923991151afda5126bc1b9bf387fc1ffce4bed14b527bd48f3fd7829 not found: ID does not exist"
Nov 04 03:29:41.015414 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:41.015345    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f7c4f5d87-28j47"]
Nov 04 03:29:41.063373 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:41.063335    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6f7c4f5d87-28j47"]
Nov 04 03:29:41.533885 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:29:41.533856    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="667a921d-333d-4244-8ea8-f0a60692a5f8" path="/var/lib/kubelet/pods/667a921d-333d-4244-8ea8-f0a60692a5f8/volumes"
Nov 04 03:29:48.565586 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:29:48.565494    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:30:00.174697 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:00.174660    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29896050-tnvsb"]
Nov 04 03:30:00.175251 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:00.175231    5609 cpu_manager.go:414] "containerMap: removing container" podUID="667a921d-333d-4244-8ea8-f0a60692a5f8" containerName="authorino"
Nov 04 03:30:00.175337 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:00.175254    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="667a921d-333d-4244-8ea8-f0a60692a5f8" containerName="authorino"
Nov 04 03:30:00.175391 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:00.175370    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="667a921d-333d-4244-8ea8-f0a60692a5f8" containerName="authorino"
Nov 04 03:30:00.178710 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:00.178687    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896050-tnvsb"
Nov 04 03:30:00.229991 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:00.229950    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x9cnx\""
Nov 04 03:30:00.279613 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:00.279580    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29896050-tnvsb"]
Nov 04 03:30:00.377823 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:00.377794    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qj6l9\" (UniqueName: \"kubernetes.io/projected/4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be-kube-api-access-qj6l9\") pod \"maas-api-key-cleanup-29896050-tnvsb\" (UID: \"4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896050-tnvsb"
Nov 04 03:30:00.378017 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:00.377843    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be-maas-api-service-ca\") pod \"maas-api-key-cleanup-29896050-tnvsb\" (UID: \"4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896050-tnvsb"
Nov 04 03:30:00.478442 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:00.478387    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be-maas-api-service-ca\") pod \"maas-api-key-cleanup-29896050-tnvsb\" (UID: \"4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896050-tnvsb"
Nov 04 03:30:00.478645 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:00.478534    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qj6l9\" (UniqueName: \"kubernetes.io/projected/4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be-kube-api-access-qj6l9\") pod \"maas-api-key-cleanup-29896050-tnvsb\" (UID: \"4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896050-tnvsb"
Nov 04 03:30:00.479016 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:00.478993    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be-maas-api-service-ca\") pod \"maas-api-key-cleanup-29896050-tnvsb\" (UID: \"4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896050-tnvsb"
Nov 04 03:30:00.537268 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:00.537242    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qj6l9\" (UniqueName: \"kubernetes.io/projected/4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be-kube-api-access-qj6l9\") pod \"maas-api-key-cleanup-29896050-tnvsb\" (UID: \"4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896050-tnvsb"
Nov 04 03:30:00.791355 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:00.791262    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896050-tnvsb"
Nov 04 03:30:01.166833 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:01.166755    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29896050-tnvsb"]
Nov 04 03:30:01.172460 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:30:01.172427    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bbc8aa3_684b_4aa5_ab34_e444aaa4a5be.slice/crio-197316a147941df8aa3b6a6b6900a86315b24f7196e04719be3d5a15ca256fa3 WatchSource:0}: Error finding container 197316a147941df8aa3b6a6b6900a86315b24f7196e04719be3d5a15ca256fa3: Status 404 returned error can't find the container with id 197316a147941df8aa3b6a6b6900a86315b24f7196e04719be3d5a15ca256fa3
Nov 04 03:30:01.983516 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:01.983482    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896050-tnvsb" event={"ID":"4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be","Type":"ContainerStarted","Data":"197316a147941df8aa3b6a6b6900a86315b24f7196e04719be3d5a15ca256fa3"}
Nov 04 03:30:09.025500 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:09.025472    5609 generic.go:356] "Generic (PLEG): container finished" podID="4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be" containerID="4d1cf8495095acf442ce51e0586d257e286fa11f0092bac612ff112481714443" exitCode=60
Nov 04 03:30:09.025867 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:09.025548    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896050-tnvsb" event={"ID":"4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be","Type":"ContainerDied","Data":"4d1cf8495095acf442ce51e0586d257e286fa11f0092bac612ff112481714443"}
Nov 04 03:30:09.025867 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:09.025732    5609 scope.go:122] "RemoveContainer" containerID="4d1cf8495095acf442ce51e0586d257e286fa11f0092bac612ff112481714443"
Nov 04 03:30:10.033108 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:10.033074    5609 generic.go:356] "Generic (PLEG): container finished" podID="4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be" containerID="a7351c9598a6b0924a11d3e350b2bc18eb1302c884fc82ac3748fb6878d1ddc0" exitCode=60
Nov 04 03:30:10.033517 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:10.033217    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896050-tnvsb" event={"ID":"4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be","Type":"ContainerDied","Data":"a7351c9598a6b0924a11d3e350b2bc18eb1302c884fc82ac3748fb6878d1ddc0"}
Nov 04 03:30:10.033517 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:10.033311    5609 scope.go:122] "RemoveContainer" containerID="4d1cf8495095acf442ce51e0586d257e286fa11f0092bac612ff112481714443"
Nov 04 03:30:10.033633 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:10.033535    5609 scope.go:122] "RemoveContainer" containerID="a7351c9598a6b0924a11d3e350b2bc18eb1302c884fc82ac3748fb6878d1ddc0"
Nov 04 03:30:10.033783 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:30:10.033765    5609 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29896050-tnvsb_odh-ai-gateway-infra(4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896050-tnvsb" podUID="4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be"
Nov 04 03:30:11.040477 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:11.040446    5609 scope.go:122] "RemoveContainer" containerID="a7351c9598a6b0924a11d3e350b2bc18eb1302c884fc82ac3748fb6878d1ddc0"
Nov 04 03:30:11.040860 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:30:11.040658    5609 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29896050-tnvsb_odh-ai-gateway-infra(4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896050-tnvsb" podUID="4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be"
Nov 04 03:30:18.560235 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:30:18.560195    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:30:20.528323 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:20.528287    5609 scope.go:122] "RemoveContainer" containerID="a7351c9598a6b0924a11d3e350b2bc18eb1302c884fc82ac3748fb6878d1ddc0"
Nov 04 03:30:21.095477 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:21.095439    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896050-tnvsb" event={"ID":"4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be","Type":"ContainerStarted","Data":"af339c7a1857f6946bb7ee52be96ff246759f97caf6632d695fba90f9bed4b38"}
Nov 04 03:30:21.207333 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:21.207285    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896050-tnvsb" podStartSLOduration=14.117376149 podStartE2EDuration="21.207273771s" totalImagesPullingTime="7.089897622s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:30:21.205284928 +0000 UTC m=+763.781937868" watchObservedRunningTime="2026-11-04 03:30:21.207273771 +0000 UTC m=+763.783926685"
Nov 04 03:30:22.102424 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:22.102372    5609 generic.go:356] "Generic (PLEG): container finished" podID="4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be" containerID="af339c7a1857f6946bb7ee52be96ff246759f97caf6632d695fba90f9bed4b38" exitCode=60
Nov 04 03:30:22.102835 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:22.102447    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896050-tnvsb" event={"ID":"4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be","Type":"ContainerDied","Data":"af339c7a1857f6946bb7ee52be96ff246759f97caf6632d695fba90f9bed4b38"}
Nov 04 03:30:22.102835 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:22.102521    5609 scope.go:122] "RemoveContainer" containerID="a7351c9598a6b0924a11d3e350b2bc18eb1302c884fc82ac3748fb6878d1ddc0"
Nov 04 03:30:22.102835 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:22.102803    5609 scope.go:122] "RemoveContainer" containerID="af339c7a1857f6946bb7ee52be96ff246759f97caf6632d695fba90f9bed4b38"
Nov 04 03:30:22.103048 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:30:22.103026    5609 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29896050-tnvsb_odh-ai-gateway-infra(4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896050-tnvsb" podUID="4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be"
Nov 04 03:30:22.222007 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:22.221974    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29896050-tnvsb"]
Nov 04 03:30:23.237774 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:23.237751    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896050-tnvsb"
Nov 04 03:30:23.289281 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:23.289258    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be-kube-api-access-qj6l9\" (UniqueName: \"kubernetes.io/projected/4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be-kube-api-access-qj6l9\") pod \"4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be\" (UID: \"4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be\") "
Nov 04 03:30:23.289442 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:23.289393    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be-maas-api-service-ca\") pod \"4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be\" (UID: \"4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be\") "
Nov 04 03:30:23.289717 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:23.289697    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be-maas-api-service-ca" pod "4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be" (UID: "4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:30:23.291373 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:23.291347    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be-kube-api-access-qj6l9" pod "4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be" (UID: "4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be"). InnerVolumeSpecName "kube-api-access-qj6l9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:30:23.390764 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:23.390697    5609 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be-maas-api-service-ca\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:30:23.390764 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:23.390721    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qj6l9\" (UniqueName: \"kubernetes.io/projected/4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be-kube-api-access-qj6l9\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:30:24.115482 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:24.115448    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896050-tnvsb"
Nov 04 03:30:24.115482 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:24.115472    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896050-tnvsb" event={"ID":"4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be","Type":"ContainerDied","Data":"197316a147941df8aa3b6a6b6900a86315b24f7196e04719be3d5a15ca256fa3"}
Nov 04 03:30:24.115756 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:24.115535    5609 scope.go:122] "RemoveContainer" containerID="af339c7a1857f6946bb7ee52be96ff246759f97caf6632d695fba90f9bed4b38"
Nov 04 03:30:24.271956 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:24.271922    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29896050-tnvsb"]
Nov 04 03:30:24.275677 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:24.275653    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29896050-tnvsb"]
Nov 04 03:30:25.534323 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:25.534291    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be" path="/var/lib/kubelet/pods/4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be/volumes"
Nov 04 03:30:47.841474 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:47.841442    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp"]
Nov 04 03:30:47.841938 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:47.841827    5609 cpu_manager.go:414] "containerMap: removing container" podUID="4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be" containerName="cleanup"
Nov 04 03:30:47.841938 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:47.841838    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be" containerName="cleanup"
Nov 04 03:30:47.841938 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:47.841863    5609 cpu_manager.go:414] "containerMap: removing container" podUID="4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be" containerName="cleanup"
Nov 04 03:30:47.841938 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:47.841869    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be" containerName="cleanup"
Nov 04 03:30:47.841938 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:47.841879    5609 cpu_manager.go:414] "containerMap: removing container" podUID="4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be" containerName="cleanup"
Nov 04 03:30:47.841938 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:47.841883    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be" containerName="cleanup"
Nov 04 03:30:47.841938 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:47.841939    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be" containerName="cleanup"
Nov 04 03:30:47.842143 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:47.841950    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be" containerName="cleanup"
Nov 04 03:30:47.847944 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:47.847922    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp"
Nov 04 03:30:47.933885 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:47.933859    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-fba151-openshift-default-dockercfg-v44lg\""
Nov 04 03:30:47.949660 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:47.949625    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp"]
Nov 04 03:30:48.010287 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.010256    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/477d4138-273f-4c56-b997-d81f053d4f34-credential-socket\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp\" (UID: \"477d4138-273f-4c56-b997-d81f053d4f34\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp"
Nov 04 03:30:48.010503 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.010297    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/477d4138-273f-4c56-b997-d81f053d4f34-istio-token\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp\" (UID: \"477d4138-273f-4c56-b997-d81f053d4f34\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp"
Nov 04 03:30:48.010503 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.010317    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/477d4138-273f-4c56-b997-d81f053d4f34-workload-certs\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp\" (UID: \"477d4138-273f-4c56-b997-d81f053d4f34\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp"
Nov 04 03:30:48.010503 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.010382    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjhc6\" (UniqueName: \"kubernetes.io/projected/477d4138-273f-4c56-b997-d81f053d4f34-kube-api-access-xjhc6\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp\" (UID: \"477d4138-273f-4c56-b997-d81f053d4f34\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp"
Nov 04 03:30:48.010618 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.010533    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/477d4138-273f-4c56-b997-d81f053d4f34-istiod-ca-cert\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp\" (UID: \"477d4138-273f-4c56-b997-d81f053d4f34\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp"
Nov 04 03:30:48.010618 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.010569    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/477d4138-273f-4c56-b997-d81f053d4f34-istio-data\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp\" (UID: \"477d4138-273f-4c56-b997-d81f053d4f34\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp"
Nov 04 03:30:48.010618 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.010590    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/477d4138-273f-4c56-b997-d81f053d4f34-istio-podinfo\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp\" (UID: \"477d4138-273f-4c56-b997-d81f053d4f34\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp"
Nov 04 03:30:48.010618 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.010608    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/477d4138-273f-4c56-b997-d81f053d4f34-istio-envoy\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp\" (UID: \"477d4138-273f-4c56-b997-d81f053d4f34\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp"
Nov 04 03:30:48.010751 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.010731    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/477d4138-273f-4c56-b997-d81f053d4f34-workload-socket\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp\" (UID: \"477d4138-273f-4c56-b997-d81f053d4f34\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp"
Nov 04 03:30:48.111440 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.111342    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/477d4138-273f-4c56-b997-d81f053d4f34-workload-socket\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp\" (UID: \"477d4138-273f-4c56-b997-d81f053d4f34\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp"
Nov 04 03:30:48.111593 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.111439    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/477d4138-273f-4c56-b997-d81f053d4f34-credential-socket\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp\" (UID: \"477d4138-273f-4c56-b997-d81f053d4f34\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp"
Nov 04 03:30:48.111593 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.111477    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/477d4138-273f-4c56-b997-d81f053d4f34-istio-token\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp\" (UID: \"477d4138-273f-4c56-b997-d81f053d4f34\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp"
Nov 04 03:30:48.111593 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.111497    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/477d4138-273f-4c56-b997-d81f053d4f34-workload-certs\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp\" (UID: \"477d4138-273f-4c56-b997-d81f053d4f34\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp"
Nov 04 03:30:48.111593 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.111515    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjhc6\" (UniqueName: \"kubernetes.io/projected/477d4138-273f-4c56-b997-d81f053d4f34-kube-api-access-xjhc6\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp\" (UID: \"477d4138-273f-4c56-b997-d81f053d4f34\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp"
Nov 04 03:30:48.111593 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.111533    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/477d4138-273f-4c56-b997-d81f053d4f34-istiod-ca-cert\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp\" (UID: \"477d4138-273f-4c56-b997-d81f053d4f34\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp"
Nov 04 03:30:48.111593 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.111567    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/477d4138-273f-4c56-b997-d81f053d4f34-istio-data\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp\" (UID: \"477d4138-273f-4c56-b997-d81f053d4f34\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp"
Nov 04 03:30:48.111593 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.111589    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/477d4138-273f-4c56-b997-d81f053d4f34-istio-podinfo\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp\" (UID: \"477d4138-273f-4c56-b997-d81f053d4f34\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp"
Nov 04 03:30:48.111945 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.111765    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/477d4138-273f-4c56-b997-d81f053d4f34-istio-envoy\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp\" (UID: \"477d4138-273f-4c56-b997-d81f053d4f34\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp"
Nov 04 03:30:48.111945 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.111843    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/477d4138-273f-4c56-b997-d81f053d4f34-workload-socket\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp\" (UID: \"477d4138-273f-4c56-b997-d81f053d4f34\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp"
Nov 04 03:30:48.111945 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.111894    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/477d4138-273f-4c56-b997-d81f053d4f34-credential-socket\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp\" (UID: \"477d4138-273f-4c56-b997-d81f053d4f34\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp"
Nov 04 03:30:48.111945 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.111929    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/477d4138-273f-4c56-b997-d81f053d4f34-istio-data\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp\" (UID: \"477d4138-273f-4c56-b997-d81f053d4f34\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp"
Nov 04 03:30:48.112124 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.112019    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/477d4138-273f-4c56-b997-d81f053d4f34-workload-certs\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp\" (UID: \"477d4138-273f-4c56-b997-d81f053d4f34\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp"
Nov 04 03:30:48.112297 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.112272    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/477d4138-273f-4c56-b997-d81f053d4f34-istiod-ca-cert\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp\" (UID: \"477d4138-273f-4c56-b997-d81f053d4f34\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp"
Nov 04 03:30:48.113959 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.113935    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/477d4138-273f-4c56-b997-d81f053d4f34-istio-envoy\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp\" (UID: \"477d4138-273f-4c56-b997-d81f053d4f34\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp"
Nov 04 03:30:48.114188 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.114170    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/477d4138-273f-4c56-b997-d81f053d4f34-istio-podinfo\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp\" (UID: \"477d4138-273f-4c56-b997-d81f053d4f34\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp"
Nov 04 03:30:48.170169 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.170131    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/477d4138-273f-4c56-b997-d81f053d4f34-istio-token\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp\" (UID: \"477d4138-273f-4c56-b997-d81f053d4f34\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp"
Nov 04 03:30:48.170986 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.170964    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjhc6\" (UniqueName: \"kubernetes.io/projected/477d4138-273f-4c56-b997-d81f053d4f34-kube-api-access-xjhc6\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp\" (UID: \"477d4138-273f-4c56-b997-d81f053d4f34\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp"
Nov 04 03:30:48.180067 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.180047    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf"]
Nov 04 03:30:48.180564 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.180550    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="4bbc8aa3-684b-4aa5-ab34-e444aaa4a5be" containerName="cleanup"
Nov 04 03:30:48.185525 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.185508    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf"
Nov 04 03:30:48.235549 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.235520    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-82c724-openshift-default-dockercfg-42cx7\""
Nov 04 03:30:48.244047 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.244015    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf"]
Nov 04 03:30:48.314088 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.314059    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-workload-socket\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf\" (UID: \"f0e29029-1a41-41a2-bfa8-b9ecd49be65b\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf"
Nov 04 03:30:48.314088 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.314091    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-istio-token\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf\" (UID: \"f0e29029-1a41-41a2-bfa8-b9ecd49be65b\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf"
Nov 04 03:30:48.314273 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.314115    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-workload-certs\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf\" (UID: \"f0e29029-1a41-41a2-bfa8-b9ecd49be65b\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf"
Nov 04 03:30:48.314273 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.314236    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-credential-socket\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf\" (UID: \"f0e29029-1a41-41a2-bfa8-b9ecd49be65b\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf"
Nov 04 03:30:48.314273 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.314265    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-istio-data\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf\" (UID: \"f0e29029-1a41-41a2-bfa8-b9ecd49be65b\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf"
Nov 04 03:30:48.314374 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.314338    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gg85c\" (UniqueName: \"kubernetes.io/projected/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-kube-api-access-gg85c\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf\" (UID: \"f0e29029-1a41-41a2-bfa8-b9ecd49be65b\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf"
Nov 04 03:30:48.314419 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.314391    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-istio-podinfo\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf\" (UID: \"f0e29029-1a41-41a2-bfa8-b9ecd49be65b\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf"
Nov 04 03:30:48.314502 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.314485    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-istio-envoy\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf\" (UID: \"f0e29029-1a41-41a2-bfa8-b9ecd49be65b\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf"
Nov 04 03:30:48.314547 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.314535    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-istiod-ca-cert\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf\" (UID: \"f0e29029-1a41-41a2-bfa8-b9ecd49be65b\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf"
Nov 04 03:30:48.415456 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.415356    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-credential-socket\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf\" (UID: \"f0e29029-1a41-41a2-bfa8-b9ecd49be65b\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf"
Nov 04 03:30:48.415456 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.415392    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-istio-data\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf\" (UID: \"f0e29029-1a41-41a2-bfa8-b9ecd49be65b\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf"
Nov 04 03:30:48.415644 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.415457    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gg85c\" (UniqueName: \"kubernetes.io/projected/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-kube-api-access-gg85c\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf\" (UID: \"f0e29029-1a41-41a2-bfa8-b9ecd49be65b\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf"
Nov 04 03:30:48.415644 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.415499    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-istio-podinfo\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf\" (UID: \"f0e29029-1a41-41a2-bfa8-b9ecd49be65b\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf"
Nov 04 03:30:48.415644 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.415622    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-istio-envoy\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf\" (UID: \"f0e29029-1a41-41a2-bfa8-b9ecd49be65b\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf"
Nov 04 03:30:48.415766 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.415707    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-istiod-ca-cert\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf\" (UID: \"f0e29029-1a41-41a2-bfa8-b9ecd49be65b\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf"
Nov 04 03:30:48.415766 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.415742    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-workload-socket\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf\" (UID: \"f0e29029-1a41-41a2-bfa8-b9ecd49be65b\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf"
Nov 04 03:30:48.415863 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.415768    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-istio-token\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf\" (UID: \"f0e29029-1a41-41a2-bfa8-b9ecd49be65b\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf"
Nov 04 03:30:48.415863 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.415776    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-credential-socket\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf\" (UID: \"f0e29029-1a41-41a2-bfa8-b9ecd49be65b\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf"
Nov 04 03:30:48.415863 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.415807    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-workload-certs\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf\" (UID: \"f0e29029-1a41-41a2-bfa8-b9ecd49be65b\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf"
Nov 04 03:30:48.416037 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.416007    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-istio-data\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf\" (UID: \"f0e29029-1a41-41a2-bfa8-b9ecd49be65b\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf"
Nov 04 03:30:48.416094 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.416040    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-workload-socket\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf\" (UID: \"f0e29029-1a41-41a2-bfa8-b9ecd49be65b\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf"
Nov 04 03:30:48.416173 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.416154    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-workload-certs\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf\" (UID: \"f0e29029-1a41-41a2-bfa8-b9ecd49be65b\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf"
Nov 04 03:30:48.416490 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.416465    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-istiod-ca-cert\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf\" (UID: \"f0e29029-1a41-41a2-bfa8-b9ecd49be65b\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf"
Nov 04 03:30:48.417904 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.417881    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-istio-envoy\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf\" (UID: \"f0e29029-1a41-41a2-bfa8-b9ecd49be65b\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf"
Nov 04 03:30:48.418116 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.418101    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-istio-podinfo\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf\" (UID: \"f0e29029-1a41-41a2-bfa8-b9ecd49be65b\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf"
Nov 04 03:30:48.461174 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.461135    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp"
Nov 04 03:30:48.474335 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.474153    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-istio-token\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf\" (UID: \"f0e29029-1a41-41a2-bfa8-b9ecd49be65b\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf"
Nov 04 03:30:48.474335 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.474305    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gg85c\" (UniqueName: \"kubernetes.io/projected/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-kube-api-access-gg85c\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf\" (UID: \"f0e29029-1a41-41a2-bfa8-b9ecd49be65b\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf"
Nov 04 03:30:48.499643 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.499616    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf"
Nov 04 03:30:48.578089 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:30:48.577909    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:30:48.886963 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.886932    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp"]
Nov 04 03:30:48.890002 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.889981    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf"]
Nov 04 03:30:48.891874 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:30:48.891847    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod477d4138_273f_4c56_b997_d81f053d4f34.slice/crio-4b7bbcc86aa0e8288552edefd7066e814eb9c72756a334a6b16dbceb05311600 WatchSource:0}: Error finding container 4b7bbcc86aa0e8288552edefd7066e814eb9c72756a334a6b16dbceb05311600: Status 404 returned error can't find the container with id 4b7bbcc86aa0e8288552edefd7066e814eb9c72756a334a6b16dbceb05311600
Nov 04 03:30:48.894035 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.894000    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:30:48.894110 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.894055    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:30:48.894110 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.894088    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:30:48.932551 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:30:48.932529    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0e29029_1a41_41a2_bfa8_b9ecd49be65b.slice/crio-187981a426d05b0b2d37529db6fbb698f5c206f811807758e26e6b57ed5f82e6 WatchSource:0}: Error finding container 187981a426d05b0b2d37529db6fbb698f5c206f811807758e26e6b57ed5f82e6: Status 404 returned error can't find the container with id 187981a426d05b0b2d37529db6fbb698f5c206f811807758e26e6b57ed5f82e6
Nov 04 03:30:48.935050 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.935025    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:30:48.935109 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.935082    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:30:48.935147 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:48.935110    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:30:49.259860 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:49.259824    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp" event={"ID":"477d4138-273f-4c56-b997-d81f053d4f34","Type":"ContainerStarted","Data":"fbb601ddfb0d9b3a25224652928d30b5f719e668dc1d4bbdc33994f18d7759e5"}
Nov 04 03:30:49.260050 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:49.259894    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp" event={"ID":"477d4138-273f-4c56-b997-d81f053d4f34","Type":"ContainerStarted","Data":"4b7bbcc86aa0e8288552edefd7066e814eb9c72756a334a6b16dbceb05311600"}
Nov 04 03:30:49.261654 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:49.261627    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf" event={"ID":"f0e29029-1a41-41a2-bfa8-b9ecd49be65b","Type":"ContainerStarted","Data":"63c87b7065c116b4a13ea75398bfbb0af83e9d050e31ef475c8507eed3dd73ec"}
Nov 04 03:30:49.261753 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:49.261670    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf" event={"ID":"f0e29029-1a41-41a2-bfa8-b9ecd49be65b","Type":"ContainerStarted","Data":"187981a426d05b0b2d37529db6fbb698f5c206f811807758e26e6b57ed5f82e6"}
Nov 04 03:30:49.375825 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:49.375778    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp" podStartSLOduration=2.375763834 podStartE2EDuration="2.375763834s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:30:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:30:49.3745584 +0000 UTC m=+791.951211310" watchObservedRunningTime="2026-11-04 03:30:49.375763834 +0000 UTC m=+791.952416832"
Nov 04 03:30:49.461568 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:49.461531    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp"
Nov 04 03:30:49.466385 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:49.466361    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp"
Nov 04 03:30:49.493847 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:49.493156    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf" podStartSLOduration=1.4931383440000001 podStartE2EDuration="1.493138344s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:30:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:30:49.489776779 +0000 UTC m=+792.066429707" watchObservedRunningTime="2026-11-04 03:30:49.493138344 +0000 UTC m=+792.069791258"
Nov 04 03:30:49.500577 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:49.500540    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf"
Nov 04 03:30:49.506695 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:49.506663    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf"
Nov 04 03:30:50.268071 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:50.268033    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp"
Nov 04 03:30:50.268548 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:50.268241    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf"
Nov 04 03:30:50.269048 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:50.269029    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf"
Nov 04 03:30:50.269163 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:50.269074    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp"
Nov 04 03:30:55.652197 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:55.652164    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb"]
Nov 04 03:30:55.656026 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:55.656006    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb"
Nov 04 03:30:55.699038 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:55.698943    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8003391e-2385-45df-b9cf-30d5c564a0c6-istio-podinfo\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb\" (UID: \"8003391e-2385-45df-b9cf-30d5c564a0c6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb"
Nov 04 03:30:55.699292 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:55.699270    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8003391e-2385-45df-b9cf-30d5c564a0c6-credential-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb\" (UID: \"8003391e-2385-45df-b9cf-30d5c564a0c6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb"
Nov 04 03:30:55.699499 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:55.699480    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8003391e-2385-45df-b9cf-30d5c564a0c6-workload-certs\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb\" (UID: \"8003391e-2385-45df-b9cf-30d5c564a0c6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb"
Nov 04 03:30:55.699684 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:55.699667    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rc87z\" (UniqueName: \"kubernetes.io/projected/8003391e-2385-45df-b9cf-30d5c564a0c6-kube-api-access-rc87z\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb\" (UID: \"8003391e-2385-45df-b9cf-30d5c564a0c6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb"
Nov 04 03:30:55.699877 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:55.699859    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8003391e-2385-45df-b9cf-30d5c564a0c6-istio-data\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb\" (UID: \"8003391e-2385-45df-b9cf-30d5c564a0c6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb"
Nov 04 03:30:55.700050 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:55.700035    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8003391e-2385-45df-b9cf-30d5c564a0c6-istio-envoy\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb\" (UID: \"8003391e-2385-45df-b9cf-30d5c564a0c6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb"
Nov 04 03:30:55.700188 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:55.700175    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8003391e-2385-45df-b9cf-30d5c564a0c6-istio-token\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb\" (UID: \"8003391e-2385-45df-b9cf-30d5c564a0c6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb"
Nov 04 03:30:55.700324 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:55.700311    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8003391e-2385-45df-b9cf-30d5c564a0c6-workload-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb\" (UID: \"8003391e-2385-45df-b9cf-30d5c564a0c6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb"
Nov 04 03:30:55.700489 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:55.700470    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8003391e-2385-45df-b9cf-30d5c564a0c6-istiod-ca-cert\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb\" (UID: \"8003391e-2385-45df-b9cf-30d5c564a0c6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb"
Nov 04 03:30:55.707904 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:55.707787    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-ee1947-openshift-default-dockercfg-2xn5j\""
Nov 04 03:30:55.759548 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:55.759511    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb"]
Nov 04 03:30:55.801690 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:55.801653    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8003391e-2385-45df-b9cf-30d5c564a0c6-istio-podinfo\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb\" (UID: \"8003391e-2385-45df-b9cf-30d5c564a0c6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb"
Nov 04 03:30:55.801853 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:55.801710    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8003391e-2385-45df-b9cf-30d5c564a0c6-credential-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb\" (UID: \"8003391e-2385-45df-b9cf-30d5c564a0c6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb"
Nov 04 03:30:55.801853 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:55.801738    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8003391e-2385-45df-b9cf-30d5c564a0c6-workload-certs\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb\" (UID: \"8003391e-2385-45df-b9cf-30d5c564a0c6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb"
Nov 04 03:30:55.801853 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:55.801784    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rc87z\" (UniqueName: \"kubernetes.io/projected/8003391e-2385-45df-b9cf-30d5c564a0c6-kube-api-access-rc87z\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb\" (UID: \"8003391e-2385-45df-b9cf-30d5c564a0c6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb"
Nov 04 03:30:55.801853 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:55.801839    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8003391e-2385-45df-b9cf-30d5c564a0c6-istio-data\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb\" (UID: \"8003391e-2385-45df-b9cf-30d5c564a0c6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb"
Nov 04 03:30:55.802076 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:55.801885    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8003391e-2385-45df-b9cf-30d5c564a0c6-istio-envoy\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb\" (UID: \"8003391e-2385-45df-b9cf-30d5c564a0c6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb"
Nov 04 03:30:55.802076 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:55.801914    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8003391e-2385-45df-b9cf-30d5c564a0c6-istio-token\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb\" (UID: \"8003391e-2385-45df-b9cf-30d5c564a0c6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb"
Nov 04 03:30:55.802076 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:55.801971    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8003391e-2385-45df-b9cf-30d5c564a0c6-workload-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb\" (UID: \"8003391e-2385-45df-b9cf-30d5c564a0c6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb"
Nov 04 03:30:55.802076 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:55.802012    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8003391e-2385-45df-b9cf-30d5c564a0c6-istiod-ca-cert\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb\" (UID: \"8003391e-2385-45df-b9cf-30d5c564a0c6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb"
Nov 04 03:30:55.802273 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:55.802160    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8003391e-2385-45df-b9cf-30d5c564a0c6-credential-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb\" (UID: \"8003391e-2385-45df-b9cf-30d5c564a0c6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb"
Nov 04 03:30:55.802273 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:55.802238    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8003391e-2385-45df-b9cf-30d5c564a0c6-workload-certs\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb\" (UID: \"8003391e-2385-45df-b9cf-30d5c564a0c6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb"
Nov 04 03:30:55.802359 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:55.802267    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8003391e-2385-45df-b9cf-30d5c564a0c6-istio-data\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb\" (UID: \"8003391e-2385-45df-b9cf-30d5c564a0c6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb"
Nov 04 03:30:55.802434 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:55.802378    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8003391e-2385-45df-b9cf-30d5c564a0c6-workload-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb\" (UID: \"8003391e-2385-45df-b9cf-30d5c564a0c6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb"
Nov 04 03:30:55.802704 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:55.802688    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8003391e-2385-45df-b9cf-30d5c564a0c6-istiod-ca-cert\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb\" (UID: \"8003391e-2385-45df-b9cf-30d5c564a0c6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb"
Nov 04 03:30:55.804376 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:55.804354    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8003391e-2385-45df-b9cf-30d5c564a0c6-istio-envoy\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb\" (UID: \"8003391e-2385-45df-b9cf-30d5c564a0c6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb"
Nov 04 03:30:55.804588 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:55.804571    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8003391e-2385-45df-b9cf-30d5c564a0c6-istio-podinfo\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb\" (UID: \"8003391e-2385-45df-b9cf-30d5c564a0c6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb"
Nov 04 03:30:55.867924 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:55.867861    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8003391e-2385-45df-b9cf-30d5c564a0c6-istio-token\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb\" (UID: \"8003391e-2385-45df-b9cf-30d5c564a0c6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb"
Nov 04 03:30:55.868801 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:55.868739    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rc87z\" (UniqueName: \"kubernetes.io/projected/8003391e-2385-45df-b9cf-30d5c564a0c6-kube-api-access-rc87z\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb\" (UID: \"8003391e-2385-45df-b9cf-30d5c564a0c6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb"
Nov 04 03:30:55.970018 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:55.969981    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb"
Nov 04 03:30:56.405123 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:56.405032    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb"]
Nov 04 03:30:56.410355 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:30:56.410328    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8003391e_2385_45df_b9cf_30d5c564a0c6.slice/crio-5129b03e2db47aaf16dbc7ba6fa1f67a67e7ab2302b8105feda5bf36591c5934 WatchSource:0}: Error finding container 5129b03e2db47aaf16dbc7ba6fa1f67a67e7ab2302b8105feda5bf36591c5934: Status 404 returned error can't find the container with id 5129b03e2db47aaf16dbc7ba6fa1f67a67e7ab2302b8105feda5bf36591c5934
Nov 04 03:30:56.412500 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:56.412464    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:30:56.412593 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:56.412546    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:30:56.412593 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:56.412574    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:30:57.309361 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:57.309326    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb" event={"ID":"8003391e-2385-45df-b9cf-30d5c564a0c6","Type":"ContainerStarted","Data":"f17f6a6c1fa388ab7f94d92ad013bd9cd3d57fcb3fc464810251de0251554873"}
Nov 04 03:30:57.309871 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:57.309387    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb" event={"ID":"8003391e-2385-45df-b9cf-30d5c564a0c6","Type":"ContainerStarted","Data":"5129b03e2db47aaf16dbc7ba6fa1f67a67e7ab2302b8105feda5bf36591c5934"}
Nov 04 03:30:57.477595 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:57.477547    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb" podStartSLOduration=2.477533793 podStartE2EDuration="2.477533793s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:30:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:30:57.474846665 +0000 UTC m=+800.051499593" watchObservedRunningTime="2026-11-04 03:30:57.477533793 +0000 UTC m=+800.054186706"
Nov 04 03:30:57.970212 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:57.970179    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb"
Nov 04 03:30:57.975169 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:57.975145    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb"
Nov 04 03:30:58.315636 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:58.315547    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb"
Nov 04 03:30:58.316507 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:58.316486    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb"
Nov 04 03:30:59.973380 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:59.973342    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x"]
Nov 04 03:30:59.977473 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:30:59.977452    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x"
Nov 04 03:31:00.038425 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:00.034918    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-092f8c-openshift-default-dockercfg-9bd4m\""
Nov 04 03:31:00.062421 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:00.061036    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/66120169-8cf1-4166-93b9-2ae671872165-istio-token\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x\" (UID: \"66120169-8cf1-4166-93b9-2ae671872165\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x"
Nov 04 03:31:00.062421 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:00.061087    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/66120169-8cf1-4166-93b9-2ae671872165-credential-socket\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x\" (UID: \"66120169-8cf1-4166-93b9-2ae671872165\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x"
Nov 04 03:31:00.062421 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:00.061117    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/66120169-8cf1-4166-93b9-2ae671872165-workload-certs\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x\" (UID: \"66120169-8cf1-4166-93b9-2ae671872165\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x"
Nov 04 03:31:00.062421 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:00.061155    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/66120169-8cf1-4166-93b9-2ae671872165-workload-socket\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x\" (UID: \"66120169-8cf1-4166-93b9-2ae671872165\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x"
Nov 04 03:31:00.062421 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:00.061182    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/66120169-8cf1-4166-93b9-2ae671872165-istio-podinfo\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x\" (UID: \"66120169-8cf1-4166-93b9-2ae671872165\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x"
Nov 04 03:31:00.062421 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:00.061231    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jphnr\" (UniqueName: \"kubernetes.io/projected/66120169-8cf1-4166-93b9-2ae671872165-kube-api-access-jphnr\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x\" (UID: \"66120169-8cf1-4166-93b9-2ae671872165\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x"
Nov 04 03:31:00.062421 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:00.061265    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/66120169-8cf1-4166-93b9-2ae671872165-istiod-ca-cert\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x\" (UID: \"66120169-8cf1-4166-93b9-2ae671872165\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x"
Nov 04 03:31:00.062421 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:00.061608    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/66120169-8cf1-4166-93b9-2ae671872165-istio-data\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x\" (UID: \"66120169-8cf1-4166-93b9-2ae671872165\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x"
Nov 04 03:31:00.062421 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:00.061686    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/66120169-8cf1-4166-93b9-2ae671872165-istio-envoy\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x\" (UID: \"66120169-8cf1-4166-93b9-2ae671872165\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x"
Nov 04 03:31:00.083501 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:00.083468    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x"]
Nov 04 03:31:00.162650 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:00.162607    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/66120169-8cf1-4166-93b9-2ae671872165-workload-socket\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x\" (UID: \"66120169-8cf1-4166-93b9-2ae671872165\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x"
Nov 04 03:31:00.162843 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:00.162661    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/66120169-8cf1-4166-93b9-2ae671872165-istio-podinfo\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x\" (UID: \"66120169-8cf1-4166-93b9-2ae671872165\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x"
Nov 04 03:31:00.162843 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:00.162708    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jphnr\" (UniqueName: \"kubernetes.io/projected/66120169-8cf1-4166-93b9-2ae671872165-kube-api-access-jphnr\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x\" (UID: \"66120169-8cf1-4166-93b9-2ae671872165\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x"
Nov 04 03:31:00.162843 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:00.162742    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/66120169-8cf1-4166-93b9-2ae671872165-istiod-ca-cert\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x\" (UID: \"66120169-8cf1-4166-93b9-2ae671872165\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x"
Nov 04 03:31:00.162843 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:00.162826    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/66120169-8cf1-4166-93b9-2ae671872165-istio-data\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x\" (UID: \"66120169-8cf1-4166-93b9-2ae671872165\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x"
Nov 04 03:31:00.163060 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:00.162879    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/66120169-8cf1-4166-93b9-2ae671872165-istio-envoy\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x\" (UID: \"66120169-8cf1-4166-93b9-2ae671872165\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x"
Nov 04 03:31:00.163060 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:00.162913    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/66120169-8cf1-4166-93b9-2ae671872165-istio-token\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x\" (UID: \"66120169-8cf1-4166-93b9-2ae671872165\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x"
Nov 04 03:31:00.163060 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:00.162944    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/66120169-8cf1-4166-93b9-2ae671872165-credential-socket\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x\" (UID: \"66120169-8cf1-4166-93b9-2ae671872165\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x"
Nov 04 03:31:00.163060 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:00.162971    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/66120169-8cf1-4166-93b9-2ae671872165-workload-certs\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x\" (UID: \"66120169-8cf1-4166-93b9-2ae671872165\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x"
Nov 04 03:31:00.163238 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:00.163089    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/66120169-8cf1-4166-93b9-2ae671872165-workload-socket\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x\" (UID: \"66120169-8cf1-4166-93b9-2ae671872165\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x"
Nov 04 03:31:00.165703 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:00.165327    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/66120169-8cf1-4166-93b9-2ae671872165-credential-socket\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x\" (UID: \"66120169-8cf1-4166-93b9-2ae671872165\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x"
Nov 04 03:31:00.169120 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:00.169094    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/66120169-8cf1-4166-93b9-2ae671872165-workload-certs\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x\" (UID: \"66120169-8cf1-4166-93b9-2ae671872165\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x"
Nov 04 03:31:00.169348 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:00.169310    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/66120169-8cf1-4166-93b9-2ae671872165-istiod-ca-cert\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x\" (UID: \"66120169-8cf1-4166-93b9-2ae671872165\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x"
Nov 04 03:31:00.169946 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:00.169925    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/66120169-8cf1-4166-93b9-2ae671872165-istio-podinfo\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x\" (UID: \"66120169-8cf1-4166-93b9-2ae671872165\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x"
Nov 04 03:31:00.173661 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:00.173638    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/66120169-8cf1-4166-93b9-2ae671872165-istio-data\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x\" (UID: \"66120169-8cf1-4166-93b9-2ae671872165\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x"
Nov 04 03:31:00.183532 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:00.183505    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/66120169-8cf1-4166-93b9-2ae671872165-istio-envoy\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x\" (UID: \"66120169-8cf1-4166-93b9-2ae671872165\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x"
Nov 04 03:31:00.221832 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:00.221792    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/66120169-8cf1-4166-93b9-2ae671872165-istio-token\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x\" (UID: \"66120169-8cf1-4166-93b9-2ae671872165\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x"
Nov 04 03:31:00.221971 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:00.221856    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jphnr\" (UniqueName: \"kubernetes.io/projected/66120169-8cf1-4166-93b9-2ae671872165-kube-api-access-jphnr\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x\" (UID: \"66120169-8cf1-4166-93b9-2ae671872165\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x"
Nov 04 03:31:00.291599 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:00.291514    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x"
Nov 04 03:31:00.725382 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:00.725345    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x"]
Nov 04 03:31:00.730269 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:31:00.730240    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66120169_8cf1_4166_93b9_2ae671872165.slice/crio-45c4c35cb9a9bbf1a798a0a61f89fd582d52c37763814ab1ba7ebc660008f394 WatchSource:0}: Error finding container 45c4c35cb9a9bbf1a798a0a61f89fd582d52c37763814ab1ba7ebc660008f394: Status 404 returned error can't find the container with id 45c4c35cb9a9bbf1a798a0a61f89fd582d52c37763814ab1ba7ebc660008f394
Nov 04 03:31:00.732452 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:00.732416    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:31:00.732549 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:00.732495    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:31:00.732549 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:00.732541    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:31:00.799159 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:00.799129    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-82c724-76bcd64bff-94mnv"]
Nov 04 03:31:00.803005 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:00.802984    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-82c724-76bcd64bff-94mnv"
Nov 04 03:31:00.854947 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:00.854770    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w3-82c724\""
Nov 04 03:31:00.854947 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:00.854914    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w3-82c724-dockercfg-lckcd\""
Nov 04 03:31:00.858892 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:00.858865    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-82c724-76bcd64bff-94mnv"]
Nov 04 03:31:00.870472 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:00.870447    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xkb4\" (UniqueName: \"kubernetes.io/projected/d8178561-e4cd-488a-a88b-c62dfa4fee20-kube-api-access-4xkb4\") pod \"payload-pre-processing-e2e-worker-w3-82c724-76bcd64bff-94mnv\" (UID: \"d8178561-e4cd-488a-a88b-c62dfa4fee20\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-82c724-76bcd64bff-94mnv"
Nov 04 03:31:00.870611 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:00.870525    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8178561-e4cd-488a-a88b-c62dfa4fee20-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-82c724-76bcd64bff-94mnv\" (UID: \"d8178561-e4cd-488a-a88b-c62dfa4fee20\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-82c724-76bcd64bff-94mnv"
Nov 04 03:31:00.971997 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:00.971950    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xkb4\" (UniqueName: \"kubernetes.io/projected/d8178561-e4cd-488a-a88b-c62dfa4fee20-kube-api-access-4xkb4\") pod \"payload-pre-processing-e2e-worker-w3-82c724-76bcd64bff-94mnv\" (UID: \"d8178561-e4cd-488a-a88b-c62dfa4fee20\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-82c724-76bcd64bff-94mnv"
Nov 04 03:31:00.972183 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:00.972035    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8178561-e4cd-488a-a88b-c62dfa4fee20-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-82c724-76bcd64bff-94mnv\" (UID: \"d8178561-e4cd-488a-a88b-c62dfa4fee20\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-82c724-76bcd64bff-94mnv"
Nov 04 03:31:00.973164 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:00.973136    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8178561-e4cd-488a-a88b-c62dfa4fee20-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-82c724-76bcd64bff-94mnv\" (UID: \"d8178561-e4cd-488a-a88b-c62dfa4fee20\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-82c724-76bcd64bff-94mnv"
Nov 04 03:31:01.001646 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:01.001577    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-82c724-6d4b6d5bb8-8cdsz"]
Nov 04 03:31:01.005434 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:01.005414    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-82c724-6d4b6d5bb8-8cdsz"
Nov 04 03:31:01.057680 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:01.057645    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xkb4\" (UniqueName: \"kubernetes.io/projected/d8178561-e4cd-488a-a88b-c62dfa4fee20-kube-api-access-4xkb4\") pod \"payload-pre-processing-e2e-worker-w3-82c724-76bcd64bff-94mnv\" (UID: \"d8178561-e4cd-488a-a88b-c62dfa4fee20\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-82c724-76bcd64bff-94mnv"
Nov 04 03:31:01.071299 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:01.069733    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-82c724-6d4b6d5bb8-8cdsz"]
Nov 04 03:31:01.072986 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:01.072956    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d29f5b1d-29ba-4f10-b17d-d2c8ccd4cd0d-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-82c724-6d4b6d5bb8-8cdsz\" (UID: \"d29f5b1d-29ba-4f10-b17d-d2c8ccd4cd0d\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-82c724-6d4b6d5bb8-8cdsz"
Nov 04 03:31:01.073133 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:01.073008    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27wzl\" (UniqueName: \"kubernetes.io/projected/d29f5b1d-29ba-4f10-b17d-d2c8ccd4cd0d-kube-api-access-27wzl\") pod \"payload-processing-e2e-worker-w3-82c724-6d4b6d5bb8-8cdsz\" (UID: \"d29f5b1d-29ba-4f10-b17d-d2c8ccd4cd0d\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-82c724-6d4b6d5bb8-8cdsz"
Nov 04 03:31:01.119112 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:01.119067    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-82c724-76bcd64bff-94mnv"
Nov 04 03:31:01.174763 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:01.174649    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d29f5b1d-29ba-4f10-b17d-d2c8ccd4cd0d-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-82c724-6d4b6d5bb8-8cdsz\" (UID: \"d29f5b1d-29ba-4f10-b17d-d2c8ccd4cd0d\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-82c724-6d4b6d5bb8-8cdsz"
Nov 04 03:31:01.174763 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:01.174711    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27wzl\" (UniqueName: \"kubernetes.io/projected/d29f5b1d-29ba-4f10-b17d-d2c8ccd4cd0d-kube-api-access-27wzl\") pod \"payload-processing-e2e-worker-w3-82c724-6d4b6d5bb8-8cdsz\" (UID: \"d29f5b1d-29ba-4f10-b17d-d2c8ccd4cd0d\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-82c724-6d4b6d5bb8-8cdsz"
Nov 04 03:31:01.176127 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:01.176095    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d29f5b1d-29ba-4f10-b17d-d2c8ccd4cd0d-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-82c724-6d4b6d5bb8-8cdsz\" (UID: \"d29f5b1d-29ba-4f10-b17d-d2c8ccd4cd0d\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-82c724-6d4b6d5bb8-8cdsz"
Nov 04 03:31:01.233640 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:01.233605    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-27wzl\" (UniqueName: \"kubernetes.io/projected/d29f5b1d-29ba-4f10-b17d-d2c8ccd4cd0d-kube-api-access-27wzl\") pod \"payload-processing-e2e-worker-w3-82c724-6d4b6d5bb8-8cdsz\" (UID: \"d29f5b1d-29ba-4f10-b17d-d2c8ccd4cd0d\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-82c724-6d4b6d5bb8-8cdsz"
Nov 04 03:31:01.319646 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:01.319552    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-82c724-6d4b6d5bb8-8cdsz"
Nov 04 03:31:01.337164 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:01.337132    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x" event={"ID":"66120169-8cf1-4166-93b9-2ae671872165","Type":"ContainerStarted","Data":"cb7dfa2d744d7fc29afec6722ed2b80825b2dcb49cfec75ad23d31ada434df12"}
Nov 04 03:31:01.337295 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:01.337232    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x" event={"ID":"66120169-8cf1-4166-93b9-2ae671872165","Type":"ContainerStarted","Data":"45c4c35cb9a9bbf1a798a0a61f89fd582d52c37763814ab1ba7ebc660008f394"}
Nov 04 03:31:01.455713 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:01.455662    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x" podStartSLOduration=2.45564224 podStartE2EDuration="2.45564224s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:30:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:31:01.453995616 +0000 UTC m=+804.030648528" watchObservedRunningTime="2026-11-04 03:31:01.45564224 +0000 UTC m=+804.032295155"
Nov 04 03:31:01.503554 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:01.503519    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-82c724-76bcd64bff-94mnv"]
Nov 04 03:31:01.508752 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:31:01.508718    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8178561_e4cd_488a_a88b_c62dfa4fee20.slice/crio-be72a982ca52618b86ca64b206796c64721c36883ee90e9c796278111ddb38b2 WatchSource:0}: Error finding container be72a982ca52618b86ca64b206796c64721c36883ee90e9c796278111ddb38b2: Status 404 returned error can't find the container with id be72a982ca52618b86ca64b206796c64721c36883ee90e9c796278111ddb38b2
Nov 04 03:31:01.709832 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:01.709801    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-82c724-6d4b6d5bb8-8cdsz"]
Nov 04 03:31:01.715258 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:31:01.715232    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd29f5b1d_29ba_4f10_b17d_d2c8ccd4cd0d.slice/crio-426b6289c8c968574e36762d4f871486854286efdcccdbd3d16abceb266ddc77 WatchSource:0}: Error finding container 426b6289c8c968574e36762d4f871486854286efdcccdbd3d16abceb266ddc77: Status 404 returned error can't find the container with id 426b6289c8c968574e36762d4f871486854286efdcccdbd3d16abceb266ddc77
Nov 04 03:31:02.291692 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:02.291659    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x"
Nov 04 03:31:02.296973 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:02.296899    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x"
Nov 04 03:31:02.344250 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:02.344216    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-82c724-6d4b6d5bb8-8cdsz" event={"ID":"d29f5b1d-29ba-4f10-b17d-d2c8ccd4cd0d","Type":"ContainerStarted","Data":"426b6289c8c968574e36762d4f871486854286efdcccdbd3d16abceb266ddc77"}
Nov 04 03:31:02.345414 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:02.345378    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-82c724-76bcd64bff-94mnv" event={"ID":"d8178561-e4cd-488a-a88b-c62dfa4fee20","Type":"ContainerStarted","Data":"be72a982ca52618b86ca64b206796c64721c36883ee90e9c796278111ddb38b2"}
Nov 04 03:31:02.345681 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:02.345669    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x"
Nov 04 03:31:02.346541 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:02.346520    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x"
Nov 04 03:31:05.400929 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:05.400888    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-82c724-6d4b6d5bb8-8cdsz" event={"ID":"d29f5b1d-29ba-4f10-b17d-d2c8ccd4cd0d","Type":"ContainerStarted","Data":"80a4001a89dcc6ad0ebc1b898028f90abc3944e22b3f8c7f151b83b47b2a6c50"}
Nov 04 03:31:05.401394 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:05.401069    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-82c724-6d4b6d5bb8-8cdsz"
Nov 04 03:31:05.402420 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:05.402387    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-82c724-76bcd64bff-94mnv" event={"ID":"d8178561-e4cd-488a-a88b-c62dfa4fee20","Type":"ContainerStarted","Data":"a5eec47bf85e1624e0ff01006d6065d67e0b0bab7fa3a686ed85e2936f6cd691"}
Nov 04 03:31:05.402547 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:05.402529    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-82c724-76bcd64bff-94mnv"
Nov 04 03:31:05.510806 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:05.510756    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w3-82c724-6d4b6d5bb8-8cdsz" podStartSLOduration=2.33052206 podStartE2EDuration="5.510741412s" totalImagesPullingTime="3.180219352s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:31:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:31:05.510126582 +0000 UTC m=+808.086779496" watchObservedRunningTime="2026-11-04 03:31:05.510741412 +0000 UTC m=+808.087394352"
Nov 04 03:31:05.621894 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:05.621834    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-82c724-76bcd64bff-94mnv" podStartSLOduration=2.353928151 podStartE2EDuration="5.621817765s" totalImagesPullingTime="3.267889614s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:31:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:31:05.619280064 +0000 UTC m=+808.195932979" watchObservedRunningTime="2026-11-04 03:31:05.621817765 +0000 UTC m=+808.198470682"
Nov 04 03:31:08.166567 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:08.166526    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-ee1947-74f4dc77d6-z7f9w"]
Nov 04 03:31:08.171091 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:08.171071    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-ee1947-74f4dc77d6-z7f9w"
Nov 04 03:31:08.247569 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:08.247541    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w1-ee1947\""
Nov 04 03:31:08.262379 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:08.262345    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae56bf38-1c41-4c65-96c1-c723def99c82-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-ee1947-74f4dc77d6-z7f9w\" (UID: \"ae56bf38-1c41-4c65-96c1-c723def99c82\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-ee1947-74f4dc77d6-z7f9w"
Nov 04 03:31:08.262529 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:08.262391    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8k82m\" (UniqueName: \"kubernetes.io/projected/ae56bf38-1c41-4c65-96c1-c723def99c82-kube-api-access-8k82m\") pod \"payload-processing-e2e-worker-w1-ee1947-74f4dc77d6-z7f9w\" (UID: \"ae56bf38-1c41-4c65-96c1-c723def99c82\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-ee1947-74f4dc77d6-z7f9w"
Nov 04 03:31:08.268828 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:08.268807    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w1-ee1947-dockercfg-m78fb\""
Nov 04 03:31:08.274089 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:08.274051    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-ee1947-74f4dc77d6-z7f9w"]
Nov 04 03:31:08.363272 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:08.363241    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8k82m\" (UniqueName: \"kubernetes.io/projected/ae56bf38-1c41-4c65-96c1-c723def99c82-kube-api-access-8k82m\") pod \"payload-processing-e2e-worker-w1-ee1947-74f4dc77d6-z7f9w\" (UID: \"ae56bf38-1c41-4c65-96c1-c723def99c82\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-ee1947-74f4dc77d6-z7f9w"
Nov 04 03:31:08.363458 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:08.363362    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae56bf38-1c41-4c65-96c1-c723def99c82-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-ee1947-74f4dc77d6-z7f9w\" (UID: \"ae56bf38-1c41-4c65-96c1-c723def99c82\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-ee1947-74f4dc77d6-z7f9w"
Nov 04 03:31:08.364008 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:08.363990    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae56bf38-1c41-4c65-96c1-c723def99c82-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-ee1947-74f4dc77d6-z7f9w\" (UID: \"ae56bf38-1c41-4c65-96c1-c723def99c82\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-ee1947-74f4dc77d6-z7f9w"
Nov 04 03:31:08.421515 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:08.421441    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8k82m\" (UniqueName: \"kubernetes.io/projected/ae56bf38-1c41-4c65-96c1-c723def99c82-kube-api-access-8k82m\") pod \"payload-processing-e2e-worker-w1-ee1947-74f4dc77d6-z7f9w\" (UID: \"ae56bf38-1c41-4c65-96c1-c723def99c82\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-ee1947-74f4dc77d6-z7f9w"
Nov 04 03:31:08.485659 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:08.485623    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-ee1947-74f4dc77d6-z7f9w"
Nov 04 03:31:08.869878 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:08.869844    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-ee1947-74f4dc77d6-z7f9w"]
Nov 04 03:31:08.874146 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:31:08.874122    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae56bf38_1c41_4c65_96c1_c723def99c82.slice/crio-0a1f9b3ed39ccf9ca5f749c3714529e9223729fcba53530ecdf1afd272c1291c WatchSource:0}: Error finding container 0a1f9b3ed39ccf9ca5f749c3714529e9223729fcba53530ecdf1afd272c1291c: Status 404 returned error can't find the container with id 0a1f9b3ed39ccf9ca5f749c3714529e9223729fcba53530ecdf1afd272c1291c
Nov 04 03:31:09.427415 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:09.427365    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-ee1947-74f4dc77d6-z7f9w" event={"ID":"ae56bf38-1c41-4c65-96c1-c723def99c82","Type":"ContainerStarted","Data":"51e74f49acfa1da2c0e73b118d6ed1905e190ca76c4eb77010c26dff2d81e1ad"}
Nov 04 03:31:09.427797 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:09.427443    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-ee1947-74f4dc77d6-z7f9w" event={"ID":"ae56bf38-1c41-4c65-96c1-c723def99c82","Type":"ContainerStarted","Data":"0a1f9b3ed39ccf9ca5f749c3714529e9223729fcba53530ecdf1afd272c1291c"}
Nov 04 03:31:09.427797 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:09.427466    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w1-ee1947-74f4dc77d6-z7f9w"
Nov 04 03:31:09.537689 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:09.537643    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w1-ee1947-74f4dc77d6-z7f9w" podStartSLOduration=1.5376308939999999 podStartE2EDuration="1.537630894s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:31:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:31:09.537308591 +0000 UTC m=+812.113961505" watchObservedRunningTime="2026-11-04 03:31:09.537630894 +0000 UTC m=+812.114283808"
Nov 04 03:31:10.705755 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:10.705697    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-50429e-7bfb595764-n2l4c"]
Nov 04 03:31:10.711087 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:10.710748    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-50429e-7bfb595764-n2l4c"
Nov 04 03:31:10.761950 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:10.761918    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-50429e-7bfb595764-n2l4c"]
Nov 04 03:31:10.812778 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:10.812739    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-50429e-dockercfg-qlgdz\""
Nov 04 03:31:10.813632 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:10.813117    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-50429e\""
Nov 04 03:31:10.814087 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:10.814059    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfh9w\" (UniqueName: \"kubernetes.io/projected/295e9e28-b9ff-4dd2-b440-5dfd9190bd9a-kube-api-access-dfh9w\") pod \"payload-processing-e2e-worker-w4-50429e-7bfb595764-n2l4c\" (UID: \"295e9e28-b9ff-4dd2-b440-5dfd9190bd9a\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-50429e-7bfb595764-n2l4c"
Nov 04 03:31:10.814363 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:10.814331    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/295e9e28-b9ff-4dd2-b440-5dfd9190bd9a-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-50429e-7bfb595764-n2l4c\" (UID: \"295e9e28-b9ff-4dd2-b440-5dfd9190bd9a\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-50429e-7bfb595764-n2l4c"
Nov 04 03:31:10.915170 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:10.915128    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfh9w\" (UniqueName: \"kubernetes.io/projected/295e9e28-b9ff-4dd2-b440-5dfd9190bd9a-kube-api-access-dfh9w\") pod \"payload-processing-e2e-worker-w4-50429e-7bfb595764-n2l4c\" (UID: \"295e9e28-b9ff-4dd2-b440-5dfd9190bd9a\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-50429e-7bfb595764-n2l4c"
Nov 04 03:31:10.915361 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:10.915213    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/295e9e28-b9ff-4dd2-b440-5dfd9190bd9a-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-50429e-7bfb595764-n2l4c\" (UID: \"295e9e28-b9ff-4dd2-b440-5dfd9190bd9a\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-50429e-7bfb595764-n2l4c"
Nov 04 03:31:10.916102 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:10.916074    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/295e9e28-b9ff-4dd2-b440-5dfd9190bd9a-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-50429e-7bfb595764-n2l4c\" (UID: \"295e9e28-b9ff-4dd2-b440-5dfd9190bd9a\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-50429e-7bfb595764-n2l4c"
Nov 04 03:31:10.983184 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:10.983089    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfh9w\" (UniqueName: \"kubernetes.io/projected/295e9e28-b9ff-4dd2-b440-5dfd9190bd9a-kube-api-access-dfh9w\") pod \"payload-processing-e2e-worker-w4-50429e-7bfb595764-n2l4c\" (UID: \"295e9e28-b9ff-4dd2-b440-5dfd9190bd9a\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-50429e-7bfb595764-n2l4c"
Nov 04 03:31:11.038633 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:11.038589    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-50429e-7bfb595764-n2l4c"
Nov 04 03:31:11.510474 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:11.504196    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-50429e-7bfb595764-n2l4c"]
Nov 04 03:31:11.543747 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:31:11.541545    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod295e9e28_b9ff_4dd2_b440_5dfd9190bd9a.slice/crio-41cbc06852a7e28e7ff921cb6d5ba4434e3476292e3360beeac6df74b0b34755 WatchSource:0}: Error finding container 41cbc06852a7e28e7ff921cb6d5ba4434e3476292e3360beeac6df74b0b34755: Status 404 returned error can't find the container with id 41cbc06852a7e28e7ff921cb6d5ba4434e3476292e3360beeac6df74b0b34755
Nov 04 03:31:12.469021 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:12.468458    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-50429e-7bfb595764-n2l4c" event={"ID":"295e9e28-b9ff-4dd2-b440-5dfd9190bd9a","Type":"ContainerStarted","Data":"daadbd46ba630116dd54c5ed9d2c9029e1e6aabf3407335278f64b719f3ec978"}
Nov 04 03:31:12.469021 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:12.468525    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-50429e-7bfb595764-n2l4c" event={"ID":"295e9e28-b9ff-4dd2-b440-5dfd9190bd9a","Type":"ContainerStarted","Data":"41cbc06852a7e28e7ff921cb6d5ba4434e3476292e3360beeac6df74b0b34755"}
Nov 04 03:31:12.469021 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:12.468659    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-50429e-7bfb595764-n2l4c"
Nov 04 03:31:12.607426 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:12.606443    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-50429e-7bfb595764-n2l4c" podStartSLOduration=2.606422268 podStartE2EDuration="2.606422268s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:31:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:31:12.580146556 +0000 UTC m=+815.156799471" watchObservedRunningTime="2026-11-04 03:31:12.606422268 +0000 UTC m=+815.183075180"
Nov 04 03:31:16.412203 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:16.412167    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-82c724-76bcd64bff-94mnv"
Nov 04 03:31:16.412628 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:16.412438    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-82c724-6d4b6d5bb8-8cdsz"
Nov 04 03:31:18.560856 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:31:18.560773    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:31:20.269025 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:20.268987    5609 patch_prober.go:29] interesting pod/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Nov 04 03:31:20.269477 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:20.269057    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf" podUID="f0e29029-1a41-41a2-bfa8-b9ecd49be65b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Nov 04 03:31:20.269477 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:20.269081    5609 patch_prober.go:29] interesting pod/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Nov 04 03:31:20.269477 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:20.269115    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp" podUID="477d4138-273f-4c56-b997-d81f053d4f34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Nov 04 03:31:20.458745 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:20.458715    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w1-ee1947-74f4dc77d6-z7f9w"
Nov 04 03:31:20.523574 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:20.523494    5609 generic.go:356] "Generic (PLEG): container finished" podID="f0e29029-1a41-41a2-bfa8-b9ecd49be65b" containerID="63c87b7065c116b4a13ea75398bfbb0af83e9d050e31ef475c8507eed3dd73ec" exitCode=0
Nov 04 03:31:20.523729 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:20.523573    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf" event={"ID":"f0e29029-1a41-41a2-bfa8-b9ecd49be65b","Type":"ContainerDied","Data":"63c87b7065c116b4a13ea75398bfbb0af83e9d050e31ef475c8507eed3dd73ec"}
Nov 04 03:31:20.523992 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:20.523973    5609 scope.go:122] "RemoveContainer" containerID="63c87b7065c116b4a13ea75398bfbb0af83e9d050e31ef475c8507eed3dd73ec"
Nov 04 03:31:20.525512 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:20.525475    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:31:20.525641 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:20.525526    5609 generic.go:356] "Generic (PLEG): container finished" podID="477d4138-273f-4c56-b997-d81f053d4f34" containerID="fbb601ddfb0d9b3a25224652928d30b5f719e668dc1d4bbdc33994f18d7759e5" exitCode=0
Nov 04 03:31:20.525641 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:20.525562    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:31:20.525641 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:20.525603    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp" event={"ID":"477d4138-273f-4c56-b997-d81f053d4f34","Type":"ContainerDied","Data":"fbb601ddfb0d9b3a25224652928d30b5f719e668dc1d4bbdc33994f18d7759e5"}
Nov 04 03:31:20.525802 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:20.525609    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:31:20.525992 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:20.525976    5609 scope.go:122] "RemoveContainer" containerID="fbb601ddfb0d9b3a25224652928d30b5f719e668dc1d4bbdc33994f18d7759e5"
Nov 04 03:31:20.527304 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:20.527275    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:31:20.527428 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:20.527335    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:31:20.527428 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:20.527374    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:31:21.543056 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:21.543028    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp" event={"ID":"477d4138-273f-4c56-b997-d81f053d4f34","Type":"ContainerStarted","Data":"7d204598c80f1adf2f988bf8ca8d258be052856ca62149e7ec42a238e7bac9c4"}
Nov 04 03:31:21.545325 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:21.545301    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf" event={"ID":"f0e29029-1a41-41a2-bfa8-b9ecd49be65b","Type":"ContainerStarted","Data":"1c512c3da901aab3287dc3832b04f5cc472266a9e36df8c69b56b2a82e7b7e75"}
Nov 04 03:31:22.462020 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:22.461985    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp"
Nov 04 03:31:22.462295 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:22.462274    5609 patch_prober.go:29] interesting pod/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Nov 04 03:31:22.462345 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:22.462320    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp" podUID="477d4138-273f-4c56-b997-d81f053d4f34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Nov 04 03:31:22.506426 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:22.505731    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf"
Nov 04 03:31:22.506426 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:22.505854    5609 patch_prober.go:29] interesting pod/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Nov 04 03:31:22.506426 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:22.505896    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf" podUID="f0e29029-1a41-41a2-bfa8-b9ecd49be65b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Nov 04 03:31:23.462410 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:23.462375    5609 patch_prober.go:29] interesting pod/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Nov 04 03:31:23.462816 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:23.462449    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp" podUID="477d4138-273f-4c56-b997-d81f053d4f34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Nov 04 03:31:23.477463 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:23.477441    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-50429e-7bfb595764-n2l4c"
Nov 04 03:31:23.503872 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:23.503754    5609 patch_prober.go:29] interesting pod/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Nov 04 03:31:23.503872 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:23.503815    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf" podUID="f0e29029-1a41-41a2-bfa8-b9ecd49be65b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Nov 04 03:31:24.470974 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:24.470935    5609 patch_prober.go:29] interesting pod/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Nov 04 03:31:24.471589 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:24.471556    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp" podUID="477d4138-273f-4c56-b997-d81f053d4f34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Nov 04 03:31:24.524424 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:24.520610    5609 patch_prober.go:29] interesting pod/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Nov 04 03:31:24.540548 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:24.539518    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf" podUID="f0e29029-1a41-41a2-bfa8-b9ecd49be65b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Nov 04 03:31:24.748479 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:24.748163    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-092f8c-679c8b7dbc-mcm7c"]
Nov 04 03:31:24.761426 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:24.760258    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-092f8c-679c8b7dbc-mcm7c"
Nov 04 03:31:24.848424 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:24.823083    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-092f8c-679c8b7dbc-mcm7c"]
Nov 04 03:31:24.859895 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:24.858960    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-092f8c-dockercfg-d8bqt\""
Nov 04 03:31:24.860130 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:24.859058    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-092f8c\""
Nov 04 03:31:24.969424 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:24.952119    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e3ba585-0eb0-44bb-955c-3b07f75123f9-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-092f8c-679c8b7dbc-mcm7c\" (UID: \"2e3ba585-0eb0-44bb-955c-3b07f75123f9\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-092f8c-679c8b7dbc-mcm7c"
Nov 04 03:31:24.969713 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:24.969680    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krx9r\" (UniqueName: \"kubernetes.io/projected/2e3ba585-0eb0-44bb-955c-3b07f75123f9-kube-api-access-krx9r\") pod \"payload-pre-processing-e2e-worker-w0-092f8c-679c8b7dbc-mcm7c\" (UID: \"2e3ba585-0eb0-44bb-955c-3b07f75123f9\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-092f8c-679c8b7dbc-mcm7c"
Nov 04 03:31:24.983500 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:24.983465    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-092f8c-7f979b7686-zq4hl"]
Nov 04 03:31:25.013456 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:25.013364    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-092f8c-7f979b7686-zq4hl"]
Nov 04 03:31:25.025788 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:25.021653    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-092f8c-7f979b7686-zq4hl"
Nov 04 03:31:25.081456 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:25.080059    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lg8j\" (UniqueName: \"kubernetes.io/projected/cf09732a-b411-43d6-81dc-0240f879a783-kube-api-access-7lg8j\") pod \"payload-processing-e2e-worker-w0-092f8c-7f979b7686-zq4hl\" (UID: \"cf09732a-b411-43d6-81dc-0240f879a783\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-092f8c-7f979b7686-zq4hl"
Nov 04 03:31:25.081786 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:25.081758    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e3ba585-0eb0-44bb-955c-3b07f75123f9-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-092f8c-679c8b7dbc-mcm7c\" (UID: \"2e3ba585-0eb0-44bb-955c-3b07f75123f9\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-092f8c-679c8b7dbc-mcm7c"
Nov 04 03:31:25.081946 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:25.081927    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krx9r\" (UniqueName: \"kubernetes.io/projected/2e3ba585-0eb0-44bb-955c-3b07f75123f9-kube-api-access-krx9r\") pod \"payload-pre-processing-e2e-worker-w0-092f8c-679c8b7dbc-mcm7c\" (UID: \"2e3ba585-0eb0-44bb-955c-3b07f75123f9\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-092f8c-679c8b7dbc-mcm7c"
Nov 04 03:31:25.082071 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:25.082054    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf09732a-b411-43d6-81dc-0240f879a783-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-092f8c-7f979b7686-zq4hl\" (UID: \"cf09732a-b411-43d6-81dc-0240f879a783\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-092f8c-7f979b7686-zq4hl"
Nov 04 03:31:25.083091 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:25.083071    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e3ba585-0eb0-44bb-955c-3b07f75123f9-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-092f8c-679c8b7dbc-mcm7c\" (UID: \"2e3ba585-0eb0-44bb-955c-3b07f75123f9\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-092f8c-679c8b7dbc-mcm7c"
Nov 04 03:31:25.220074 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:25.220033    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf09732a-b411-43d6-81dc-0240f879a783-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-092f8c-7f979b7686-zq4hl\" (UID: \"cf09732a-b411-43d6-81dc-0240f879a783\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-092f8c-7f979b7686-zq4hl"
Nov 04 03:31:25.232670 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:25.232623    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lg8j\" (UniqueName: \"kubernetes.io/projected/cf09732a-b411-43d6-81dc-0240f879a783-kube-api-access-7lg8j\") pod \"payload-processing-e2e-worker-w0-092f8c-7f979b7686-zq4hl\" (UID: \"cf09732a-b411-43d6-81dc-0240f879a783\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-092f8c-7f979b7686-zq4hl"
Nov 04 03:31:25.258426 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:25.234141    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf09732a-b411-43d6-81dc-0240f879a783-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-092f8c-7f979b7686-zq4hl\" (UID: \"cf09732a-b411-43d6-81dc-0240f879a783\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-092f8c-7f979b7686-zq4hl"
Nov 04 03:31:25.268564 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:25.268487    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-krx9r\" (UniqueName: \"kubernetes.io/projected/2e3ba585-0eb0-44bb-955c-3b07f75123f9-kube-api-access-krx9r\") pod \"payload-pre-processing-e2e-worker-w0-092f8c-679c8b7dbc-mcm7c\" (UID: \"2e3ba585-0eb0-44bb-955c-3b07f75123f9\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-092f8c-679c8b7dbc-mcm7c"
Nov 04 03:31:25.320140 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:25.320104    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lg8j\" (UniqueName: \"kubernetes.io/projected/cf09732a-b411-43d6-81dc-0240f879a783-kube-api-access-7lg8j\") pod \"payload-processing-e2e-worker-w0-092f8c-7f979b7686-zq4hl\" (UID: \"cf09732a-b411-43d6-81dc-0240f879a783\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-092f8c-7f979b7686-zq4hl"
Nov 04 03:31:25.376344 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:25.376257    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-092f8c-7f979b7686-zq4hl"
Nov 04 03:31:25.380600 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:25.380570    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-092f8c-679c8b7dbc-mcm7c"
Nov 04 03:31:25.463810 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:25.463773    5609 patch_prober.go:29] interesting pod/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Nov 04 03:31:25.464103 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:25.464061    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp" podUID="477d4138-273f-4c56-b997-d81f053d4f34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Nov 04 03:31:25.512426 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:25.500849    5609 patch_prober.go:29] interesting pod/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Nov 04 03:31:25.512426 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:25.500906    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf" podUID="f0e29029-1a41-41a2-bfa8-b9ecd49be65b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Nov 04 03:31:26.467950 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:26.467503    5609 patch_prober.go:29] interesting pod/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Nov 04 03:31:26.467950 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:26.467564    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp" podUID="477d4138-273f-4c56-b997-d81f053d4f34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Nov 04 03:31:26.512034 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:26.511908    5609 patch_prober.go:29] interesting pod/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Nov 04 03:31:26.512034 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:26.511967    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf" podUID="f0e29029-1a41-41a2-bfa8-b9ecd49be65b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Nov 04 03:31:26.570915 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:26.570861    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-092f8c-679c8b7dbc-mcm7c"]
Nov 04 03:31:26.680761 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:26.680725    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-092f8c-679c8b7dbc-mcm7c" event={"ID":"2e3ba585-0eb0-44bb-955c-3b07f75123f9","Type":"ContainerStarted","Data":"8696c14f1cf67a31b83cb5b19731c6dc7a2090dbd76282b2abed3f4abb22670f"}
Nov 04 03:31:26.970082 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:26.970049    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-092f8c-7f979b7686-zq4hl"]
Nov 04 03:31:26.978279 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:31:26.978239    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf09732a_b411_43d6_81dc_0240f879a783.slice/crio-1d01b311b87b9b7c035d9df76c899c81d7e3d150f1b0aa5575979b617df65831 WatchSource:0}: Error finding container 1d01b311b87b9b7c035d9df76c899c81d7e3d150f1b0aa5575979b617df65831: Status 404 returned error can't find the container with id 1d01b311b87b9b7c035d9df76c899c81d7e3d150f1b0aa5575979b617df65831
Nov 04 03:31:27.465663 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:27.465626    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp"
Nov 04 03:31:27.465945 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:27.465921    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp"
Nov 04 03:31:27.466690 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:27.466674    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp"
Nov 04 03:31:27.512715 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:27.512682    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf"
Nov 04 03:31:27.513023 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:27.513001    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf"
Nov 04 03:31:27.513789 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:27.513766    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf"
Nov 04 03:31:27.715195 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:27.715138    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-092f8c-679c8b7dbc-mcm7c" event={"ID":"2e3ba585-0eb0-44bb-955c-3b07f75123f9","Type":"ContainerStarted","Data":"e3dbebc453dee4a2c586f564b27f2ce58892cafd8a1a8d83c83d63542d940ec5"}
Nov 04 03:31:27.715723 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:27.715338    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-092f8c-679c8b7dbc-mcm7c"
Nov 04 03:31:27.717266 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:27.717236    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-092f8c-7f979b7686-zq4hl" event={"ID":"cf09732a-b411-43d6-81dc-0240f879a783","Type":"ContainerStarted","Data":"5c77b6d26398d0c21b00cdbe7bed4203097ca2ebde46ecc7a4ea6d8d1abd6630"}
Nov 04 03:31:27.717442 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:27.717289    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-092f8c-7f979b7686-zq4hl" event={"ID":"cf09732a-b411-43d6-81dc-0240f879a783","Type":"ContainerStarted","Data":"1d01b311b87b9b7c035d9df76c899c81d7e3d150f1b0aa5575979b617df65831"}
Nov 04 03:31:27.717790 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:27.717775    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-092f8c-7f979b7686-zq4hl"
Nov 04 03:31:27.941298 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:27.941245    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-092f8c-7f979b7686-zq4hl" podStartSLOduration=3.941231976 podStartE2EDuration="3.941231976s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:31:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:31:27.938757191 +0000 UTC m=+830.515410105" watchObservedRunningTime="2026-11-04 03:31:27.941231976 +0000 UTC m=+830.517884897"
Nov 04 03:31:28.071933 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:28.071838    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-092f8c-679c8b7dbc-mcm7c" podStartSLOduration=4.071823913 podStartE2EDuration="4.071823913s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:31:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:31:28.070779568 +0000 UTC m=+830.647432482" watchObservedRunningTime="2026-11-04 03:31:28.071823913 +0000 UTC m=+830.648476826"
Nov 04 03:31:34.766856 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:34.766823    5609 generic.go:356] "Generic (PLEG): container finished" podID="8003391e-2385-45df-b9cf-30d5c564a0c6" containerID="f17f6a6c1fa388ab7f94d92ad013bd9cd3d57fcb3fc464810251de0251554873" exitCode=0
Nov 04 03:31:34.767273 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:34.766872    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb" event={"ID":"8003391e-2385-45df-b9cf-30d5c564a0c6","Type":"ContainerDied","Data":"f17f6a6c1fa388ab7f94d92ad013bd9cd3d57fcb3fc464810251de0251554873"}
Nov 04 03:31:34.767273 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:34.767235    5609 scope.go:122] "RemoveContainer" containerID="f17f6a6c1fa388ab7f94d92ad013bd9cd3d57fcb3fc464810251de0251554873"
Nov 04 03:31:34.768558 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:34.768529    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:31:34.768624 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:34.768597    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:31:34.768659 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:34.768629    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:31:35.777100 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:35.777064    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb" event={"ID":"8003391e-2385-45df-b9cf-30d5c564a0c6","Type":"ContainerStarted","Data":"5a0dde2b6cbf34972d5639e23e4b21e46e2d6c59c076185c8c19f3876032a9a8"}
Nov 04 03:31:35.971009 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:35.970977    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb"
Nov 04 03:31:35.971181 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:35.971149    5609 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Nov 04 03:31:35.971224 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:35.971195    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb" podUID="8003391e-2385-45df-b9cf-30d5c564a0c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Nov 04 03:31:36.971422 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:36.971366    5609 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Nov 04 03:31:36.971811 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:36.971449    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb" podUID="8003391e-2385-45df-b9cf-30d5c564a0c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Nov 04 03:31:37.970551 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:37.970516    5609 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Nov 04 03:31:37.970725 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:37.970565    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb" podUID="8003391e-2385-45df-b9cf-30d5c564a0c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Nov 04 03:31:38.727227 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:38.727192    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-092f8c-679c8b7dbc-mcm7c"
Nov 04 03:31:38.970941 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:38.970906    5609 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Nov 04 03:31:38.971139 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:38.970967    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb" podUID="8003391e-2385-45df-b9cf-30d5c564a0c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Nov 04 03:31:39.734865 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:39.734833    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-092f8c-7f979b7686-zq4hl"
Nov 04 03:31:39.971563 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:39.970805    5609 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Nov 04 03:31:39.971563 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:39.970867    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb" podUID="8003391e-2385-45df-b9cf-30d5c564a0c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Nov 04 03:31:40.976698 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:40.976663    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb"
Nov 04 03:31:40.977185 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:40.977035    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb"
Nov 04 03:31:40.977631 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:40.977610    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb"
Nov 04 03:31:48.560882 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:31:48.560846    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:31:58.990715 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:58.990687    5609 generic.go:356] "Generic (PLEG): container finished" podID="66120169-8cf1-4166-93b9-2ae671872165" containerID="cb7dfa2d744d7fc29afec6722ed2b80825b2dcb49cfec75ad23d31ada434df12" exitCode=0
Nov 04 03:31:58.991055 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:58.990757    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x" event={"ID":"66120169-8cf1-4166-93b9-2ae671872165","Type":"ContainerDied","Data":"cb7dfa2d744d7fc29afec6722ed2b80825b2dcb49cfec75ad23d31ada434df12"}
Nov 04 03:31:58.991148 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:58.991134    5609 scope.go:122] "RemoveContainer" containerID="cb7dfa2d744d7fc29afec6722ed2b80825b2dcb49cfec75ad23d31ada434df12"
Nov 04 03:31:58.992408 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:58.992370    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:31:58.992467 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:58.992453    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:31:58.992498 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:31:58.992489    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:32:00.017349 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:32:00.017310    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x" event={"ID":"66120169-8cf1-4166-93b9-2ae671872165","Type":"ContainerStarted","Data":"2cad8d652e639b0c844641145cb8345e20739565d7838e050b2a11e4584e2313"}
Nov 04 03:32:00.304006 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:32:00.302762    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x"
Nov 04 03:32:00.305097 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:32:00.305068    5609 patch_prober.go:29] interesting pod/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Nov 04 03:32:00.305297 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:32:00.305270    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x" podUID="66120169-8cf1-4166-93b9-2ae671872165" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Nov 04 03:32:01.292887 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:32:01.292846    5609 patch_prober.go:29] interesting pod/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Nov 04 03:32:01.293523 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:32:01.293488    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x" podUID="66120169-8cf1-4166-93b9-2ae671872165" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Nov 04 03:32:02.306823 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:32:02.306785    5609 patch_prober.go:29] interesting pod/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Nov 04 03:32:02.307369 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:32:02.307336    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x" podUID="66120169-8cf1-4166-93b9-2ae671872165" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Nov 04 03:32:02.346472 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:32:02.346443    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x"
Nov 04 03:32:03.297865 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:32:03.297823    5609 patch_prober.go:29] interesting pod/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Nov 04 03:32:03.298173 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:32:03.298138    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x" podUID="66120169-8cf1-4166-93b9-2ae671872165" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Nov 04 03:32:04.295428 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:32:04.292854    5609 patch_prober.go:29] interesting pod/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Nov 04 03:32:04.295428 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:32:04.292946    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x" podUID="66120169-8cf1-4166-93b9-2ae671872165" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Nov 04 03:32:05.297243 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:32:05.297213    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x"
Nov 04 03:32:05.298422 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:32:05.298383    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x"
Nov 04 03:32:18.559541 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:32:18.559506    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:32:37.639799 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:32:37.639770    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-p9989_068619f8-7fbb-4b22-ac7d-a1f144e084c0/console-operator/2.log"
Nov 04 03:32:37.647893 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:32:37.647865    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vmw5z_21a42913-5c4e-4057-aae5-2804038f16e4/ovn-acl-logging/0.log"
Nov 04 03:32:37.650103 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:32:37.650076    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-p9989_068619f8-7fbb-4b22-ac7d-a1f144e084c0/console-operator/2.log"
Nov 04 03:32:37.658170 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:32:37.658143    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vmw5z_21a42913-5c4e-4057-aae5-2804038f16e4/ovn-acl-logging/0.log"
Nov 04 03:32:48.559936 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:32:48.559849    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:33:18.573340 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:33:18.573303    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:33:48.559875 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:33:48.559841    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:33:54.032864 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:33:54.032821    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-nqvbk-2lhwt"]
Nov 04 03:33:54.037056 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:33:54.037032    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-nqvbk-2lhwt"
Nov 04 03:33:54.092127 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:33:54.092088    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-nqvbk-2lhwt"]
Nov 04 03:33:54.134904 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:33:54.134871    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-gj69c\""
Nov 04 03:33:54.138547 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:33:54.138519    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8fjc\" (UniqueName: \"kubernetes.io/projected/8602b662-6b9f-4c9e-b924-971cab5fc508-kube-api-access-p8fjc\") pod \"periodic-gathering-nqvbk-2lhwt\" (UID: \"8602b662-6b9f-4c9e-b924-971cab5fc508\") " pod="openshift-insights/periodic-gathering-nqvbk-2lhwt"
Nov 04 03:33:54.138692 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:33:54.138668    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/8602b662-6b9f-4c9e-b924-971cab5fc508-archives-path\") pod \"periodic-gathering-nqvbk-2lhwt\" (UID: \"8602b662-6b9f-4c9e-b924-971cab5fc508\") " pod="openshift-insights/periodic-gathering-nqvbk-2lhwt"
Nov 04 03:33:54.138779 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:33:54.138761    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8602b662-6b9f-4c9e-b924-971cab5fc508-service-ca-bundle\") pod \"periodic-gathering-nqvbk-2lhwt\" (UID: \"8602b662-6b9f-4c9e-b924-971cab5fc508\") " pod="openshift-insights/periodic-gathering-nqvbk-2lhwt"
Nov 04 03:33:54.138868 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:33:54.138847    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8602b662-6b9f-4c9e-b924-971cab5fc508-trusted-ca-bundle\") pod \"periodic-gathering-nqvbk-2lhwt\" (UID: \"8602b662-6b9f-4c9e-b924-971cab5fc508\") " pod="openshift-insights/periodic-gathering-nqvbk-2lhwt"
Nov 04 03:33:54.240362 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:33:54.240326    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8fjc\" (UniqueName: \"kubernetes.io/projected/8602b662-6b9f-4c9e-b924-971cab5fc508-kube-api-access-p8fjc\") pod \"periodic-gathering-nqvbk-2lhwt\" (UID: \"8602b662-6b9f-4c9e-b924-971cab5fc508\") " pod="openshift-insights/periodic-gathering-nqvbk-2lhwt"
Nov 04 03:33:54.240561 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:33:54.240444    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/8602b662-6b9f-4c9e-b924-971cab5fc508-archives-path\") pod \"periodic-gathering-nqvbk-2lhwt\" (UID: \"8602b662-6b9f-4c9e-b924-971cab5fc508\") " pod="openshift-insights/periodic-gathering-nqvbk-2lhwt"
Nov 04 03:33:54.240561 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:33:54.240485    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8602b662-6b9f-4c9e-b924-971cab5fc508-service-ca-bundle\") pod \"periodic-gathering-nqvbk-2lhwt\" (UID: \"8602b662-6b9f-4c9e-b924-971cab5fc508\") " pod="openshift-insights/periodic-gathering-nqvbk-2lhwt"
Nov 04 03:33:54.240561 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:33:54.240520    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8602b662-6b9f-4c9e-b924-971cab5fc508-trusted-ca-bundle\") pod \"periodic-gathering-nqvbk-2lhwt\" (UID: \"8602b662-6b9f-4c9e-b924-971cab5fc508\") " pod="openshift-insights/periodic-gathering-nqvbk-2lhwt"
Nov 04 03:33:54.240978 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:33:54.240956    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/8602b662-6b9f-4c9e-b924-971cab5fc508-archives-path\") pod \"periodic-gathering-nqvbk-2lhwt\" (UID: \"8602b662-6b9f-4c9e-b924-971cab5fc508\") " pod="openshift-insights/periodic-gathering-nqvbk-2lhwt"
Nov 04 03:33:54.241151 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:33:54.241134    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8602b662-6b9f-4c9e-b924-971cab5fc508-service-ca-bundle\") pod \"periodic-gathering-nqvbk-2lhwt\" (UID: \"8602b662-6b9f-4c9e-b924-971cab5fc508\") " pod="openshift-insights/periodic-gathering-nqvbk-2lhwt"
Nov 04 03:33:54.241317 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:33:54.241300    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8602b662-6b9f-4c9e-b924-971cab5fc508-trusted-ca-bundle\") pod \"periodic-gathering-nqvbk-2lhwt\" (UID: \"8602b662-6b9f-4c9e-b924-971cab5fc508\") " pod="openshift-insights/periodic-gathering-nqvbk-2lhwt"
Nov 04 03:33:54.298600 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:33:54.298534    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8fjc\" (UniqueName: \"kubernetes.io/projected/8602b662-6b9f-4c9e-b924-971cab5fc508-kube-api-access-p8fjc\") pod \"periodic-gathering-nqvbk-2lhwt\" (UID: \"8602b662-6b9f-4c9e-b924-971cab5fc508\") " pod="openshift-insights/periodic-gathering-nqvbk-2lhwt"
Nov 04 03:33:54.349649 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:33:54.349615    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-nqvbk-2lhwt"
Nov 04 03:33:54.733757 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:33:54.733723    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-nqvbk-2lhwt"]
Nov 04 03:33:54.738681 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:33:54.738652    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8602b662_6b9f_4c9e_b924_971cab5fc508.slice/crio-cced18fb79a1a3ddb13e30cb4b48d47d1b09a32f8c8c32b251fe656cd2cd1bec WatchSource:0}: Error finding container cced18fb79a1a3ddb13e30cb4b48d47d1b09a32f8c8c32b251fe656cd2cd1bec: Status 404 returned error can't find the container with id cced18fb79a1a3ddb13e30cb4b48d47d1b09a32f8c8c32b251fe656cd2cd1bec
Nov 04 03:33:54.810117 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:33:54.810080    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-nqvbk-2lhwt" event={"ID":"8602b662-6b9f-4c9e-b924-971cab5fc508","Type":"ContainerStarted","Data":"cced18fb79a1a3ddb13e30cb4b48d47d1b09a32f8c8c32b251fe656cd2cd1bec"}
Nov 04 03:33:55.819909 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:33:55.819869    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-nqvbk-2lhwt" event={"ID":"8602b662-6b9f-4c9e-b924-971cab5fc508","Type":"ContainerStarted","Data":"8ec18df56e5332f5d1f911b87b2e9833312e3f005161c019da4a53c561a8d3aa"}
Nov 04 03:33:55.931920 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:33:55.931860    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-nqvbk-2lhwt" podStartSLOduration=2.93184471 podStartE2EDuration="2.93184471s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:33:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:33:55.929345071 +0000 UTC m=+978.505997986" watchObservedRunningTime="2026-11-04 03:33:55.93184471 +0000 UTC m=+978.508497623"
Nov 04 03:34:18.560160 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:34:18.560074    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:34:48.561359 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:34:48.561312    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:35:18.568489 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:35:18.568449    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:35:31.524569 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:31.524536    5609 generic.go:356] "Generic (PLEG): container finished" podID="8602b662-6b9f-4c9e-b924-971cab5fc508" containerID="8ec18df56e5332f5d1f911b87b2e9833312e3f005161c019da4a53c561a8d3aa" exitCode=0
Nov 04 03:35:31.525026 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:31.524605    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-nqvbk-2lhwt" event={"ID":"8602b662-6b9f-4c9e-b924-971cab5fc508","Type":"ContainerDied","Data":"8ec18df56e5332f5d1f911b87b2e9833312e3f005161c019da4a53c561a8d3aa"}
Nov 04 03:35:32.669591 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:32.669567    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-nqvbk-2lhwt"
Nov 04 03:35:32.799344 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:32.799271    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8602b662-6b9f-4c9e-b924-971cab5fc508-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8602b662-6b9f-4c9e-b924-971cab5fc508-service-ca-bundle\") pod \"8602b662-6b9f-4c9e-b924-971cab5fc508\" (UID: \"8602b662-6b9f-4c9e-b924-971cab5fc508\") "
Nov 04 03:35:32.799509 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:32.799378    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8602b662-6b9f-4c9e-b924-971cab5fc508-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/8602b662-6b9f-4c9e-b924-971cab5fc508-archives-path\") pod \"8602b662-6b9f-4c9e-b924-971cab5fc508\" (UID: \"8602b662-6b9f-4c9e-b924-971cab5fc508\") "
Nov 04 03:35:32.799509 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:32.799440    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8602b662-6b9f-4c9e-b924-971cab5fc508-kube-api-access-p8fjc\" (UniqueName: \"kubernetes.io/projected/8602b662-6b9f-4c9e-b924-971cab5fc508-kube-api-access-p8fjc\") pod \"8602b662-6b9f-4c9e-b924-971cab5fc508\" (UID: \"8602b662-6b9f-4c9e-b924-971cab5fc508\") "
Nov 04 03:35:32.799509 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:32.799474    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8602b662-6b9f-4c9e-b924-971cab5fc508-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8602b662-6b9f-4c9e-b924-971cab5fc508-trusted-ca-bundle\") pod \"8602b662-6b9f-4c9e-b924-971cab5fc508\" (UID: \"8602b662-6b9f-4c9e-b924-971cab5fc508\") "
Nov 04 03:35:32.799713 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:32.799691    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8602b662-6b9f-4c9e-b924-971cab5fc508-service-ca-bundle" pod "8602b662-6b9f-4c9e-b924-971cab5fc508" (UID: "8602b662-6b9f-4c9e-b924-971cab5fc508"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:35:32.799791 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:32.799752    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8602b662-6b9f-4c9e-b924-971cab5fc508-archives-path" pod "8602b662-6b9f-4c9e-b924-971cab5fc508" (UID: "8602b662-6b9f-4c9e-b924-971cab5fc508"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:35:32.799917 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:32.799894    5609 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8602b662-6b9f-4c9e-b924-971cab5fc508-service-ca-bundle\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:35:32.800040 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:32.800015    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8602b662-6b9f-4c9e-b924-971cab5fc508-trusted-ca-bundle" pod "8602b662-6b9f-4c9e-b924-971cab5fc508" (UID: "8602b662-6b9f-4c9e-b924-971cab5fc508"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:35:32.801791 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:32.801767    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8602b662-6b9f-4c9e-b924-971cab5fc508-kube-api-access-p8fjc" pod "8602b662-6b9f-4c9e-b924-971cab5fc508" (UID: "8602b662-6b9f-4c9e-b924-971cab5fc508"). InnerVolumeSpecName "kube-api-access-p8fjc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:35:32.900957 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:32.900921    5609 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/8602b662-6b9f-4c9e-b924-971cab5fc508-archives-path\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:35:32.900957 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:32.900950    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p8fjc\" (UniqueName: \"kubernetes.io/projected/8602b662-6b9f-4c9e-b924-971cab5fc508-kube-api-access-p8fjc\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:35:32.900957 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:32.900960    5609 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8602b662-6b9f-4c9e-b924-971cab5fc508-trusted-ca-bundle\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:35:33.538162 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:33.538126    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-nqvbk-2lhwt"
Nov 04 03:35:33.538162 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:33.538149    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-nqvbk-2lhwt" event={"ID":"8602b662-6b9f-4c9e-b924-971cab5fc508","Type":"ContainerDied","Data":"cced18fb79a1a3ddb13e30cb4b48d47d1b09a32f8c8c32b251fe656cd2cd1bec"}
Nov 04 03:35:33.538364 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:33.538188    5609 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cced18fb79a1a3ddb13e30cb4b48d47d1b09a32f8c8c32b251fe656cd2cd1bec"
Nov 04 03:35:37.577182 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:37.577153    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7370-7bea52"]
Nov 04 03:35:37.577687 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:37.577611    5609 cpu_manager.go:414] "containerMap: removing container" podUID="8602b662-6b9f-4c9e-b924-971cab5fc508" containerName="insights-gathering"
Nov 04 03:35:37.577687 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:37.577622    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8602b662-6b9f-4c9e-b924-971cab5fc508" containerName="insights-gathering"
Nov 04 03:35:37.577823 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:37.577742    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="8602b662-6b9f-4c9e-b924-971cab5fc508" containerName="insights-gathering"
Nov 04 03:35:37.581264 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:37.581245    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7370-7bea52"
Nov 04 03:35:37.631538 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:37.631515    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7370-7bea52"]
Nov 04 03:35:37.650686 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:37.650653    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7sdv\" (UniqueName: \"kubernetes.io/projected/0656e838-64c9-47f3-b866-e46520b3b0be-kube-api-access-f7sdv\") pod \"test-curl-7370-7bea52\" (UID: \"0656e838-64c9-47f3-b866-e46520b3b0be\") " pod="opendatahub/test-curl-7370-7bea52"
Nov 04 03:35:37.675472 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:37.675447    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-cccth\""
Nov 04 03:35:37.754060 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:37.754013    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7sdv\" (UniqueName: \"kubernetes.io/projected/0656e838-64c9-47f3-b866-e46520b3b0be-kube-api-access-f7sdv\") pod \"test-curl-7370-7bea52\" (UID: \"0656e838-64c9-47f3-b866-e46520b3b0be\") " pod="opendatahub/test-curl-7370-7bea52"
Nov 04 03:35:37.812732 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:37.812702    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7sdv\" (UniqueName: \"kubernetes.io/projected/0656e838-64c9-47f3-b866-e46520b3b0be-kube-api-access-f7sdv\") pod \"test-curl-7370-7bea52\" (UID: \"0656e838-64c9-47f3-b866-e46520b3b0be\") " pod="opendatahub/test-curl-7370-7bea52"
Nov 04 03:35:37.893599 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:37.893523    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7370-7bea52"
Nov 04 03:35:38.262765 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:38.262735    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7370-7bea52"]
Nov 04 03:35:38.268281 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:35:38.268254    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0656e838_64c9_47f3_b866_e46520b3b0be.slice/crio-aaf364a971a3288ba31d04d9b929fa08692ab6760ecf0b6e65b45fa412a4a8e8 WatchSource:0}: Error finding container aaf364a971a3288ba31d04d9b929fa08692ab6760ecf0b6e65b45fa412a4a8e8: Status 404 returned error can't find the container with id aaf364a971a3288ba31d04d9b929fa08692ab6760ecf0b6e65b45fa412a4a8e8
Nov 04 03:35:38.270124 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:38.270106    5609 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 03:35:38.570171 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:38.570086    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7370-7bea52" event={"ID":"0656e838-64c9-47f3-b866-e46520b3b0be","Type":"ContainerStarted","Data":"aaf364a971a3288ba31d04d9b929fa08692ab6760ecf0b6e65b45fa412a4a8e8"}
Nov 04 03:35:42.598035 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:42.597990    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7370-7bea52" event={"ID":"0656e838-64c9-47f3-b866-e46520b3b0be","Type":"ContainerStarted","Data":"cc84094928b0911f22cbe285561031ca5f50f9428a71c9636a750c345afb0403"}
Nov 04 03:35:42.716325 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:42.716265    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7370-7bea52" podStartSLOduration=2.09762922 podStartE2EDuration="5.716251961s" totalImagesPullingTime="3.618622741s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:35:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:35:42.713706025 +0000 UTC m=+1085.290358938" watchObservedRunningTime="2026-11-04 03:35:42.716251961 +0000 UTC m=+1085.292904875"
Nov 04 03:35:44.228091 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:44.228054    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7370-7bea52"]
Nov 04 03:35:44.238793 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:44.238756    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7370-7bea52"]
Nov 04 03:35:44.393497 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:44.393465    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7370-aff15b"]
Nov 04 03:35:44.394109 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:44.394089    5609 cpu_manager.go:414] "containerMap: removing container" podUID="0656e838-64c9-47f3-b866-e46520b3b0be" containerName="test-curl-7370-7bea52"
Nov 04 03:35:44.394180 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:44.394111    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0656e838-64c9-47f3-b866-e46520b3b0be" containerName="test-curl-7370-7bea52"
Nov 04 03:35:44.394222 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:44.394208    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="0656e838-64c9-47f3-b866-e46520b3b0be" containerName="test-curl-7370-7bea52"
Nov 04 03:35:44.397694 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:44.397676    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7370-aff15b"
Nov 04 03:35:44.449971 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:44.449938    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7370-aff15b"]
Nov 04 03:35:44.526184 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:44.526107    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzdpv\" (UniqueName: \"kubernetes.io/projected/2f2cc23d-ac92-4b24-b157-5858c15e8d1c-kube-api-access-dzdpv\") pod \"test-curl-7370-aff15b\" (UID: \"2f2cc23d-ac92-4b24-b157-5858c15e8d1c\") " pod="opendatahub/test-curl-7370-aff15b"
Nov 04 03:35:44.609988 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:44.609948    5609 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7370-7bea52" podUID="0656e838-64c9-47f3-b866-e46520b3b0be" containerName="test-curl-7370-7bea52" containerID="cri-o://cc84094928b0911f22cbe285561031ca5f50f9428a71c9636a750c345afb0403" gracePeriod=2
Nov 04 03:35:44.626974 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:44.626945    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzdpv\" (UniqueName: \"kubernetes.io/projected/2f2cc23d-ac92-4b24-b157-5858c15e8d1c-kube-api-access-dzdpv\") pod \"test-curl-7370-aff15b\" (UID: \"2f2cc23d-ac92-4b24-b157-5858c15e8d1c\") " pod="opendatahub/test-curl-7370-aff15b"
Nov 04 03:35:44.658794 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:35:44.658764    5609 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7370-7bea52\" is forbidden: User \"system:node:ip-10-0-128-50.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-50.us-west-2.compute.internal' and this object" podUID="0656e838-64c9-47f3-b866-e46520b3b0be" pod="opendatahub/test-curl-7370-7bea52"
Nov 04 03:35:44.685446 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:44.685417    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzdpv\" (UniqueName: \"kubernetes.io/projected/2f2cc23d-ac92-4b24-b157-5858c15e8d1c-kube-api-access-dzdpv\") pod \"test-curl-7370-aff15b\" (UID: \"2f2cc23d-ac92-4b24-b157-5858c15e8d1c\") " pod="opendatahub/test-curl-7370-aff15b"
Nov 04 03:35:44.710299 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:44.710213    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7370-aff15b"
Nov 04 03:35:45.085632 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:45.085601    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7370-aff15b"]
Nov 04 03:35:45.090493 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:35:45.090464    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2f2cc23d_ac92_4b24_b157_5858c15e8d1c.slice/crio-9bda474222c855f0d7e5c487c8f32cb42f45bd6dcba7a754997b852286480032 WatchSource:0}: Error finding container 9bda474222c855f0d7e5c487c8f32cb42f45bd6dcba7a754997b852286480032: Status 404 returned error can't find the container with id 9bda474222c855f0d7e5c487c8f32cb42f45bd6dcba7a754997b852286480032
Nov 04 03:35:45.617699 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:45.617660    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7370-aff15b" event={"ID":"2f2cc23d-ac92-4b24-b157-5858c15e8d1c","Type":"ContainerStarted","Data":"08eb5246199d469a7652571c5f743e9261fd276ee9bed2c41c0500295de86a5b"}
Nov 04 03:35:45.618273 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:45.617707    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7370-aff15b" event={"ID":"2f2cc23d-ac92-4b24-b157-5858c15e8d1c","Type":"ContainerStarted","Data":"9bda474222c855f0d7e5c487c8f32cb42f45bd6dcba7a754997b852286480032"}
Nov 04 03:35:45.727823 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:45.727768    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7370-aff15b" podStartSLOduration=1.727754808 podStartE2EDuration="1.727754808s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:35:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:35:45.725086084 +0000 UTC m=+1088.301738997" watchObservedRunningTime="2026-11-04 03:35:45.727754808 +0000 UTC m=+1088.304407722"
Nov 04 03:35:46.839336 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:46.839313    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7370-7bea52"
Nov 04 03:35:46.949233 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:46.949207    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0656e838-64c9-47f3-b866-e46520b3b0be-kube-api-access-f7sdv\" (UniqueName: \"kubernetes.io/projected/0656e838-64c9-47f3-b866-e46520b3b0be-kube-api-access-f7sdv\") pod \"0656e838-64c9-47f3-b866-e46520b3b0be\" (UID: \"0656e838-64c9-47f3-b866-e46520b3b0be\") "
Nov 04 03:35:46.951214 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:46.951182    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0656e838-64c9-47f3-b866-e46520b3b0be-kube-api-access-f7sdv" pod "0656e838-64c9-47f3-b866-e46520b3b0be" (UID: "0656e838-64c9-47f3-b866-e46520b3b0be"). InnerVolumeSpecName "kube-api-access-f7sdv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:35:47.050981 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:47.050947    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f7sdv\" (UniqueName: \"kubernetes.io/projected/0656e838-64c9-47f3-b866-e46520b3b0be-kube-api-access-f7sdv\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:35:47.356171 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:47.356090    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7370-aff15b"]
Nov 04 03:35:47.364240 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:47.364208    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7370-aff15b"]
Nov 04 03:35:47.534848 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:47.534808    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0656e838-64c9-47f3-b866-e46520b3b0be" path="/var/lib/kubelet/pods/0656e838-64c9-47f3-b866-e46520b3b0be/volumes"
Nov 04 03:35:47.633873 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:47.633785    5609 generic.go:356] "Generic (PLEG): container finished" podID="0656e838-64c9-47f3-b866-e46520b3b0be" containerID="cc84094928b0911f22cbe285561031ca5f50f9428a71c9636a750c345afb0403" exitCode=137
Nov 04 03:35:47.633873 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:47.633853    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7370-7bea52"
Nov 04 03:35:47.634092 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:47.633886    5609 scope.go:122] "RemoveContainer" containerID="cc84094928b0911f22cbe285561031ca5f50f9428a71c9636a750c345afb0403"
Nov 04 03:35:47.634092 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:47.634033    5609 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7370-aff15b" podUID="2f2cc23d-ac92-4b24-b157-5858c15e8d1c" containerName="test-curl-7370-aff15b" containerID="cri-o://08eb5246199d469a7652571c5f743e9261fd276ee9bed2c41c0500295de86a5b" gracePeriod=2
Nov 04 03:35:47.643776 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:47.643758    5609 scope.go:122] "RemoveContainer" containerID="cc84094928b0911f22cbe285561031ca5f50f9428a71c9636a750c345afb0403"
Nov 04 03:35:47.644024 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:35:47.644004    5609 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc84094928b0911f22cbe285561031ca5f50f9428a71c9636a750c345afb0403\": container with ID starting with cc84094928b0911f22cbe285561031ca5f50f9428a71c9636a750c345afb0403 not found: ID does not exist" containerID="cc84094928b0911f22cbe285561031ca5f50f9428a71c9636a750c345afb0403"
Nov 04 03:35:47.644071 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:47.644033    5609 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc84094928b0911f22cbe285561031ca5f50f9428a71c9636a750c345afb0403"} err="failed to get container status \"cc84094928b0911f22cbe285561031ca5f50f9428a71c9636a750c345afb0403\": rpc error: code = NotFound desc = could not find container \"cc84094928b0911f22cbe285561031ca5f50f9428a71c9636a750c345afb0403\": container with ID starting with cc84094928b0911f22cbe285561031ca5f50f9428a71c9636a750c345afb0403 not found: ID does not exist"
Nov 04 03:35:48.559787 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:35:48.559742    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:35:49.854312 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:49.854289    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7370-aff15b"
Nov 04 03:35:49.985594 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:49.985564    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2f2cc23d-ac92-4b24-b157-5858c15e8d1c-kube-api-access-dzdpv\" (UniqueName: \"kubernetes.io/projected/2f2cc23d-ac92-4b24-b157-5858c15e8d1c-kube-api-access-dzdpv\") pod \"2f2cc23d-ac92-4b24-b157-5858c15e8d1c\" (UID: \"2f2cc23d-ac92-4b24-b157-5858c15e8d1c\") "
Nov 04 03:35:49.987610 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:49.987586    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f2cc23d-ac92-4b24-b157-5858c15e8d1c-kube-api-access-dzdpv" pod "2f2cc23d-ac92-4b24-b157-5858c15e8d1c" (UID: "2f2cc23d-ac92-4b24-b157-5858c15e8d1c"). InnerVolumeSpecName "kube-api-access-dzdpv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:35:50.086655 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:50.086624    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dzdpv\" (UniqueName: \"kubernetes.io/projected/2f2cc23d-ac92-4b24-b157-5858c15e8d1c-kube-api-access-dzdpv\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:35:50.656999 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:50.656969    5609 generic.go:356] "Generic (PLEG): container finished" podID="2f2cc23d-ac92-4b24-b157-5858c15e8d1c" containerID="08eb5246199d469a7652571c5f743e9261fd276ee9bed2c41c0500295de86a5b" exitCode=137
Nov 04 03:35:50.657199 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:50.657046    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7370-aff15b"
Nov 04 03:35:50.657199 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:50.657077    5609 scope.go:122] "RemoveContainer" containerID="08eb5246199d469a7652571c5f743e9261fd276ee9bed2c41c0500295de86a5b"
Nov 04 03:35:50.667507 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:50.667492    5609 scope.go:122] "RemoveContainer" containerID="08eb5246199d469a7652571c5f743e9261fd276ee9bed2c41c0500295de86a5b"
Nov 04 03:35:50.667749 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:35:50.667733    5609 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08eb5246199d469a7652571c5f743e9261fd276ee9bed2c41c0500295de86a5b\": container with ID starting with 08eb5246199d469a7652571c5f743e9261fd276ee9bed2c41c0500295de86a5b not found: ID does not exist" containerID="08eb5246199d469a7652571c5f743e9261fd276ee9bed2c41c0500295de86a5b"
Nov 04 03:35:50.667800 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:50.667755    5609 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08eb5246199d469a7652571c5f743e9261fd276ee9bed2c41c0500295de86a5b"} err="failed to get container status \"08eb5246199d469a7652571c5f743e9261fd276ee9bed2c41c0500295de86a5b\": rpc error: code = NotFound desc = could not find container \"08eb5246199d469a7652571c5f743e9261fd276ee9bed2c41c0500295de86a5b\": container with ID starting with 08eb5246199d469a7652571c5f743e9261fd276ee9bed2c41c0500295de86a5b not found: ID does not exist"
Nov 04 03:35:50.714161 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:35:50.714132    5609 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7370-aff15b\" is forbidden: User \"system:node:ip-10-0-128-50.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-50.us-west-2.compute.internal' and this object" podUID="2f2cc23d-ac92-4b24-b157-5858c15e8d1c" pod="opendatahub/test-curl-7370-aff15b"
Nov 04 03:35:51.014738 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:51.014692    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk"]
Nov 04 03:35:51.015225 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:51.015210    5609 cpu_manager.go:414] "containerMap: removing container" podUID="2f2cc23d-ac92-4b24-b157-5858c15e8d1c" containerName="test-curl-7370-aff15b"
Nov 04 03:35:51.015267 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:51.015227    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2f2cc23d-ac92-4b24-b157-5858c15e8d1c" containerName="test-curl-7370-aff15b"
Nov 04 03:35:51.015317 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:51.015308    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="2f2cc23d-ac92-4b24-b157-5858c15e8d1c" containerName="test-curl-7370-aff15b"
Nov 04 03:35:51.020136 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:51.020117    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk"
Nov 04 03:35:51.087539 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:51.087494    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-a-5d4032-openshift-default-dockercfg-zq9rn\""
Nov 04 03:35:51.101107 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:51.101070    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f315bbc-cddf-41ee-b044-92914a203e58-istio-envoy\") pod \"e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk\" (UID: \"1f315bbc-cddf-41ee-b044-92914a203e58\") " pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk"
Nov 04 03:35:51.101294 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:51.101126    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f315bbc-cddf-41ee-b044-92914a203e58-istio-data\") pod \"e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk\" (UID: \"1f315bbc-cddf-41ee-b044-92914a203e58\") " pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk"
Nov 04 03:35:51.101294 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:51.101198    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f315bbc-cddf-41ee-b044-92914a203e58-istiod-ca-cert\") pod \"e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk\" (UID: \"1f315bbc-cddf-41ee-b044-92914a203e58\") " pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk"
Nov 04 03:35:51.101450 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:51.101295    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f315bbc-cddf-41ee-b044-92914a203e58-istio-podinfo\") pod \"e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk\" (UID: \"1f315bbc-cddf-41ee-b044-92914a203e58\") " pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk"
Nov 04 03:35:51.101450 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:51.101330    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f315bbc-cddf-41ee-b044-92914a203e58-workload-certs\") pod \"e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk\" (UID: \"1f315bbc-cddf-41ee-b044-92914a203e58\") " pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk"
Nov 04 03:35:51.101450 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:51.101420    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f315bbc-cddf-41ee-b044-92914a203e58-workload-socket\") pod \"e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk\" (UID: \"1f315bbc-cddf-41ee-b044-92914a203e58\") " pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk"
Nov 04 03:35:51.101612 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:51.101462    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f315bbc-cddf-41ee-b044-92914a203e58-credential-socket\") pod \"e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk\" (UID: \"1f315bbc-cddf-41ee-b044-92914a203e58\") " pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk"
Nov 04 03:35:51.101612 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:51.101497    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxlwk\" (UniqueName: \"kubernetes.io/projected/1f315bbc-cddf-41ee-b044-92914a203e58-kube-api-access-lxlwk\") pod \"e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk\" (UID: \"1f315bbc-cddf-41ee-b044-92914a203e58\") " pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk"
Nov 04 03:35:51.101612 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:51.101571    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f315bbc-cddf-41ee-b044-92914a203e58-istio-token\") pod \"e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk\" (UID: \"1f315bbc-cddf-41ee-b044-92914a203e58\") " pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk"
Nov 04 03:35:51.132359 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:51.132319    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk"]
Nov 04 03:35:51.191492 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:35:51.191443    5609 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7370-aff15b\" is forbidden: User \"system:node:ip-10-0-128-50.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-50.us-west-2.compute.internal' and this object" podUID="2f2cc23d-ac92-4b24-b157-5858c15e8d1c" pod="opendatahub/test-curl-7370-aff15b"
Nov 04 03:35:51.202428 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:51.202361    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f315bbc-cddf-41ee-b044-92914a203e58-istio-data\") pod \"e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk\" (UID: \"1f315bbc-cddf-41ee-b044-92914a203e58\") " pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk"
Nov 04 03:35:51.202611 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:51.202451    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f315bbc-cddf-41ee-b044-92914a203e58-istiod-ca-cert\") pod \"e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk\" (UID: \"1f315bbc-cddf-41ee-b044-92914a203e58\") " pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk"
Nov 04 03:35:51.202611 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:51.202503    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f315bbc-cddf-41ee-b044-92914a203e58-istio-podinfo\") pod \"e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk\" (UID: \"1f315bbc-cddf-41ee-b044-92914a203e58\") " pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk"
Nov 04 03:35:51.202611 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:51.202525    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f315bbc-cddf-41ee-b044-92914a203e58-workload-certs\") pod \"e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk\" (UID: \"1f315bbc-cddf-41ee-b044-92914a203e58\") " pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk"
Nov 04 03:35:51.202611 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:51.202589    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f315bbc-cddf-41ee-b044-92914a203e58-workload-socket\") pod \"e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk\" (UID: \"1f315bbc-cddf-41ee-b044-92914a203e58\") " pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk"
Nov 04 03:35:51.202836 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:51.202621    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f315bbc-cddf-41ee-b044-92914a203e58-credential-socket\") pod \"e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk\" (UID: \"1f315bbc-cddf-41ee-b044-92914a203e58\") " pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk"
Nov 04 03:35:51.202836 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:51.202648    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxlwk\" (UniqueName: \"kubernetes.io/projected/1f315bbc-cddf-41ee-b044-92914a203e58-kube-api-access-lxlwk\") pod \"e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk\" (UID: \"1f315bbc-cddf-41ee-b044-92914a203e58\") " pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk"
Nov 04 03:35:51.202836 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:51.202710    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f315bbc-cddf-41ee-b044-92914a203e58-istio-token\") pod \"e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk\" (UID: \"1f315bbc-cddf-41ee-b044-92914a203e58\") " pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk"
Nov 04 03:35:51.202836 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:51.202774    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f315bbc-cddf-41ee-b044-92914a203e58-istio-envoy\") pod \"e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk\" (UID: \"1f315bbc-cddf-41ee-b044-92914a203e58\") " pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk"
Nov 04 03:35:51.202988 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:51.202845    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f315bbc-cddf-41ee-b044-92914a203e58-istio-data\") pod \"e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk\" (UID: \"1f315bbc-cddf-41ee-b044-92914a203e58\") " pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk"
Nov 04 03:35:51.203105 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:51.203078    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f315bbc-cddf-41ee-b044-92914a203e58-workload-certs\") pod \"e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk\" (UID: \"1f315bbc-cddf-41ee-b044-92914a203e58\") " pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk"
Nov 04 03:35:51.203220 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:51.203178    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f315bbc-cddf-41ee-b044-92914a203e58-credential-socket\") pod \"e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk\" (UID: \"1f315bbc-cddf-41ee-b044-92914a203e58\") " pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk"
Nov 04 03:35:51.203681 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:51.203660    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f315bbc-cddf-41ee-b044-92914a203e58-istiod-ca-cert\") pod \"e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk\" (UID: \"1f315bbc-cddf-41ee-b044-92914a203e58\") " pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk"
Nov 04 03:35:51.203949 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:51.203928    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f315bbc-cddf-41ee-b044-92914a203e58-workload-socket\") pod \"e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk\" (UID: \"1f315bbc-cddf-41ee-b044-92914a203e58\") " pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk"
Nov 04 03:35:51.205123 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:51.205099    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f315bbc-cddf-41ee-b044-92914a203e58-istio-envoy\") pod \"e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk\" (UID: \"1f315bbc-cddf-41ee-b044-92914a203e58\") " pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk"
Nov 04 03:35:51.205338 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:51.205320    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f315bbc-cddf-41ee-b044-92914a203e58-istio-podinfo\") pod \"e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk\" (UID: \"1f315bbc-cddf-41ee-b044-92914a203e58\") " pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk"
Nov 04 03:35:51.263889 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:51.263866    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxlwk\" (UniqueName: \"kubernetes.io/projected/1f315bbc-cddf-41ee-b044-92914a203e58-kube-api-access-lxlwk\") pod \"e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk\" (UID: \"1f315bbc-cddf-41ee-b044-92914a203e58\") " pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk"
Nov 04 03:35:51.265143 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:51.265085    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f315bbc-cddf-41ee-b044-92914a203e58-istio-token\") pod \"e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk\" (UID: \"1f315bbc-cddf-41ee-b044-92914a203e58\") " pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk"
Nov 04 03:35:51.334039 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:51.334008    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk"
Nov 04 03:35:51.535493 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:51.535420    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2f2cc23d-ac92-4b24-b157-5858c15e8d1c" path="/var/lib/kubelet/pods/2f2cc23d-ac92-4b24-b157-5858c15e8d1c/volumes"
Nov 04 03:35:51.768952 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:51.768910    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk"]
Nov 04 03:35:51.773932 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:35:51.773902    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f315bbc_cddf_41ee_b044_92914a203e58.slice/crio-b677db0951b336fb9e9ef1abc6a38163829e3a6752f0db5ccc0eb6b1e0a1c2f1 WatchSource:0}: Error finding container b677db0951b336fb9e9ef1abc6a38163829e3a6752f0db5ccc0eb6b1e0a1c2f1: Status 404 returned error can't find the container with id b677db0951b336fb9e9ef1abc6a38163829e3a6752f0db5ccc0eb6b1e0a1c2f1
Nov 04 03:35:51.776223 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:51.776190    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:35:51.776338 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:51.776269    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:35:51.776338 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:51.776313    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:35:52.672606 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:52.672571    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk" event={"ID":"1f315bbc-cddf-41ee-b044-92914a203e58","Type":"ContainerStarted","Data":"cade2663d5901a6475aae1eb80d2a270fcca95a241dd92b627656ff6db176ea7"}
Nov 04 03:35:52.673022 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:52.672627    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk" event={"ID":"1f315bbc-cddf-41ee-b044-92914a203e58","Type":"ContainerStarted","Data":"b677db0951b336fb9e9ef1abc6a38163829e3a6752f0db5ccc0eb6b1e0a1c2f1"}
Nov 04 03:35:52.789467 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:52.789376    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk" podStartSLOduration=2.789362553 podStartE2EDuration="2.789362553s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:35:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:35:52.788174453 +0000 UTC m=+1095.364827368" watchObservedRunningTime="2026-11-04 03:35:52.789362553 +0000 UTC m=+1095.366015467"
Nov 04 03:35:53.335058 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:53.335020    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk"
Nov 04 03:35:53.340435 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:53.340382    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk"
Nov 04 03:35:53.679299 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:53.679265    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk"
Nov 04 03:35:53.680140 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:53.680120    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk"
Nov 04 03:35:59.477767 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:59.477731    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v"]
Nov 04 03:35:59.481727 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:59.481708    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v"
Nov 04 03:35:59.543107 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:59.543078    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-a-8ecbc5-openshift-default-dockercfg-ch6v6\""
Nov 04 03:35:59.588382 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:59.588344    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v"]
Nov 04 03:35:59.594902 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:59.594871    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20f015bc-d781-4317-99c3-ce697c740441-istio-envoy\") pod \"e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v\" (UID: \"20f015bc-d781-4317-99c3-ce697c740441\") " pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v"
Nov 04 03:35:59.595045 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:59.594921    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20f015bc-d781-4317-99c3-ce697c740441-istio-data\") pod \"e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v\" (UID: \"20f015bc-d781-4317-99c3-ce697c740441\") " pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v"
Nov 04 03:35:59.595085 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:59.595043    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20f015bc-d781-4317-99c3-ce697c740441-istio-token\") pod \"e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v\" (UID: \"20f015bc-d781-4317-99c3-ce697c740441\") " pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v"
Nov 04 03:35:59.595124 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:59.595086    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20f015bc-d781-4317-99c3-ce697c740441-workload-certs\") pod \"e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v\" (UID: \"20f015bc-d781-4317-99c3-ce697c740441\") " pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v"
Nov 04 03:35:59.595226 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:59.595205    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpdmq\" (UniqueName: \"kubernetes.io/projected/20f015bc-d781-4317-99c3-ce697c740441-kube-api-access-kpdmq\") pod \"e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v\" (UID: \"20f015bc-d781-4317-99c3-ce697c740441\") " pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v"
Nov 04 03:35:59.595289 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:59.595267    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20f015bc-d781-4317-99c3-ce697c740441-istiod-ca-cert\") pod \"e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v\" (UID: \"20f015bc-d781-4317-99c3-ce697c740441\") " pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v"
Nov 04 03:35:59.595447 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:59.595424    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20f015bc-d781-4317-99c3-ce697c740441-istio-podinfo\") pod \"e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v\" (UID: \"20f015bc-d781-4317-99c3-ce697c740441\") " pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v"
Nov 04 03:35:59.595508 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:59.595493    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20f015bc-d781-4317-99c3-ce697c740441-workload-socket\") pod \"e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v\" (UID: \"20f015bc-d781-4317-99c3-ce697c740441\") " pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v"
Nov 04 03:35:59.595547 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:59.595523    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20f015bc-d781-4317-99c3-ce697c740441-credential-socket\") pod \"e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v\" (UID: \"20f015bc-d781-4317-99c3-ce697c740441\") " pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v"
Nov 04 03:35:59.696862 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:59.696829    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20f015bc-d781-4317-99c3-ce697c740441-istio-podinfo\") pod \"e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v\" (UID: \"20f015bc-d781-4317-99c3-ce697c740441\") " pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v"
Nov 04 03:35:59.697030 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:59.696882    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20f015bc-d781-4317-99c3-ce697c740441-workload-socket\") pod \"e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v\" (UID: \"20f015bc-d781-4317-99c3-ce697c740441\") " pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v"
Nov 04 03:35:59.697030 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:59.696899    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20f015bc-d781-4317-99c3-ce697c740441-credential-socket\") pod \"e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v\" (UID: \"20f015bc-d781-4317-99c3-ce697c740441\") " pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v"
Nov 04 03:35:59.697030 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:59.697017    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20f015bc-d781-4317-99c3-ce697c740441-istio-envoy\") pod \"e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v\" (UID: \"20f015bc-d781-4317-99c3-ce697c740441\") " pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v"
Nov 04 03:35:59.697221 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:59.697062    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20f015bc-d781-4317-99c3-ce697c740441-istio-data\") pod \"e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v\" (UID: \"20f015bc-d781-4317-99c3-ce697c740441\") " pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v"
Nov 04 03:35:59.697221 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:59.697127    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20f015bc-d781-4317-99c3-ce697c740441-istio-token\") pod \"e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v\" (UID: \"20f015bc-d781-4317-99c3-ce697c740441\") " pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v"
Nov 04 03:35:59.697221 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:59.697174    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20f015bc-d781-4317-99c3-ce697c740441-workload-certs\") pod \"e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v\" (UID: \"20f015bc-d781-4317-99c3-ce697c740441\") " pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v"
Nov 04 03:35:59.697369 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:59.697232    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpdmq\" (UniqueName: \"kubernetes.io/projected/20f015bc-d781-4317-99c3-ce697c740441-kube-api-access-kpdmq\") pod \"e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v\" (UID: \"20f015bc-d781-4317-99c3-ce697c740441\") " pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v"
Nov 04 03:35:59.697369 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:59.697272    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20f015bc-d781-4317-99c3-ce697c740441-credential-socket\") pod \"e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v\" (UID: \"20f015bc-d781-4317-99c3-ce697c740441\") " pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v"
Nov 04 03:35:59.697369 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:59.697285    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20f015bc-d781-4317-99c3-ce697c740441-istiod-ca-cert\") pod \"e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v\" (UID: \"20f015bc-d781-4317-99c3-ce697c740441\") " pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v"
Nov 04 03:35:59.697369 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:59.697300    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20f015bc-d781-4317-99c3-ce697c740441-workload-socket\") pod \"e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v\" (UID: \"20f015bc-d781-4317-99c3-ce697c740441\") " pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v"
Nov 04 03:35:59.697604 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:59.697501    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20f015bc-d781-4317-99c3-ce697c740441-istio-data\") pod \"e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v\" (UID: \"20f015bc-d781-4317-99c3-ce697c740441\") " pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v"
Nov 04 03:35:59.697604 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:59.697570    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20f015bc-d781-4317-99c3-ce697c740441-workload-certs\") pod \"e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v\" (UID: \"20f015bc-d781-4317-99c3-ce697c740441\") " pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v"
Nov 04 03:35:59.697987 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:59.697963    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20f015bc-d781-4317-99c3-ce697c740441-istiod-ca-cert\") pod \"e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v\" (UID: \"20f015bc-d781-4317-99c3-ce697c740441\") " pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v"
Nov 04 03:35:59.699311 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:59.699290    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20f015bc-d781-4317-99c3-ce697c740441-istio-envoy\") pod \"e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v\" (UID: \"20f015bc-d781-4317-99c3-ce697c740441\") " pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v"
Nov 04 03:35:59.699511 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:59.699495    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20f015bc-d781-4317-99c3-ce697c740441-istio-podinfo\") pod \"e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v\" (UID: \"20f015bc-d781-4317-99c3-ce697c740441\") " pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v"
Nov 04 03:35:59.759991 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:59.759920    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20f015bc-d781-4317-99c3-ce697c740441-istio-token\") pod \"e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v\" (UID: \"20f015bc-d781-4317-99c3-ce697c740441\") " pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v"
Nov 04 03:35:59.771186 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:59.771147    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpdmq\" (UniqueName: \"kubernetes.io/projected/20f015bc-d781-4317-99c3-ce697c740441-kube-api-access-kpdmq\") pod \"e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v\" (UID: \"20f015bc-d781-4317-99c3-ce697c740441\") " pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v"
Nov 04 03:35:59.797626 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:35:59.797594    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v"
Nov 04 03:36:00.225700 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:00.225662    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v"]
Nov 04 03:36:00.230901 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:36:00.230875    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20f015bc_d781_4317_99c3_ce697c740441.slice/crio-9a480d6132b34046501f93fb90d6a572172aa33b41b326972571ef090ba02aeb WatchSource:0}: Error finding container 9a480d6132b34046501f93fb90d6a572172aa33b41b326972571ef090ba02aeb: Status 404 returned error can't find the container with id 9a480d6132b34046501f93fb90d6a572172aa33b41b326972571ef090ba02aeb
Nov 04 03:36:00.233455 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:00.233382    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:36:00.233559 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:00.233496    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:36:00.233559 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:00.233525    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:36:00.728805 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:00.728760    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v" event={"ID":"20f015bc-d781-4317-99c3-ce697c740441","Type":"ContainerStarted","Data":"a41c54b503aa3d6a91a1fbaf959aa2a7f1d4aaba983e4222391160c984d4677d"}
Nov 04 03:36:00.729234 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:00.728832    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v" event={"ID":"20f015bc-d781-4317-99c3-ce697c740441","Type":"ContainerStarted","Data":"9a480d6132b34046501f93fb90d6a572172aa33b41b326972571ef090ba02aeb"}
Nov 04 03:36:00.797819 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:00.797788    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v"
Nov 04 03:36:00.844662 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:00.844607    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v" podStartSLOduration=1.844593159 podStartE2EDuration="1.844593159s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:35:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:36:00.844031247 +0000 UTC m=+1103.420684161" watchObservedRunningTime="2026-11-04 03:36:00.844593159 +0000 UTC m=+1103.421246072"
Nov 04 03:36:01.805096 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:01.805065    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v"
Nov 04 03:36:02.741852 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:02.741823    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v"
Nov 04 03:36:02.742839 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:02.742816    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v"
Nov 04 03:36:07.401084 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:07.401043    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-8ecbc5-7df459b5c9-f2hdp"]
Nov 04 03:36:07.405642 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:07.405613    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-8ecbc5-7df459b5c9-f2hdp"
Nov 04 03:36:07.490976 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:07.490930    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b263e963-52bb-4426-b6e6-e0b18f1ee810-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-8ecbc5-7df459b5c9-f2hdp\" (UID: \"b263e963-52bb-4426-b6e6-e0b18f1ee810\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-8ecbc5-7df459b5c9-f2hdp"
Nov 04 03:36:07.491184 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:07.491057    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dkp6\" (UniqueName: \"kubernetes.io/projected/b263e963-52bb-4426-b6e6-e0b18f1ee810-kube-api-access-5dkp6\") pod \"payload-pre-processing-e2e-extmod-a-8ecbc5-7df459b5c9-f2hdp\" (UID: \"b263e963-52bb-4426-b6e6-e0b18f1ee810\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-8ecbc5-7df459b5c9-f2hdp"
Nov 04 03:36:07.519422 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:07.518819    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-a-8ecbc5-dockercfg-77bb6\""
Nov 04 03:36:07.523970 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:07.523926    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-a-8ecbc5\""
Nov 04 03:36:07.525529 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:07.525498    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-8ecbc5-7df459b5c9-f2hdp"]
Nov 04 03:36:07.578924 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:07.578858    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-8ecbc5-776655854d-p7qp2"]
Nov 04 03:36:07.588720 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:07.588696    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-8ecbc5-776655854d-p7qp2"
Nov 04 03:36:07.592395 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:07.592368    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b263e963-52bb-4426-b6e6-e0b18f1ee810-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-8ecbc5-7df459b5c9-f2hdp\" (UID: \"b263e963-52bb-4426-b6e6-e0b18f1ee810\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-8ecbc5-7df459b5c9-f2hdp"
Nov 04 03:36:07.592535 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:07.592489    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dkp6\" (UniqueName: \"kubernetes.io/projected/b263e963-52bb-4426-b6e6-e0b18f1ee810-kube-api-access-5dkp6\") pod \"payload-pre-processing-e2e-extmod-a-8ecbc5-7df459b5c9-f2hdp\" (UID: \"b263e963-52bb-4426-b6e6-e0b18f1ee810\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-8ecbc5-7df459b5c9-f2hdp"
Nov 04 03:36:07.593042 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:07.593019    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b263e963-52bb-4426-b6e6-e0b18f1ee810-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-8ecbc5-7df459b5c9-f2hdp\" (UID: \"b263e963-52bb-4426-b6e6-e0b18f1ee810\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-8ecbc5-7df459b5c9-f2hdp"
Nov 04 03:36:07.602222 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:07.602196    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-8ecbc5-776655854d-p7qp2"]
Nov 04 03:36:07.666501 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:07.666433    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dkp6\" (UniqueName: \"kubernetes.io/projected/b263e963-52bb-4426-b6e6-e0b18f1ee810-kube-api-access-5dkp6\") pod \"payload-pre-processing-e2e-extmod-a-8ecbc5-7df459b5c9-f2hdp\" (UID: \"b263e963-52bb-4426-b6e6-e0b18f1ee810\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-8ecbc5-7df459b5c9-f2hdp"
Nov 04 03:36:07.693839 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:07.693813    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpjgz\" (UniqueName: \"kubernetes.io/projected/f4096fc6-30ab-48db-82f5-85f9847473e0-kube-api-access-mpjgz\") pod \"payload-processing-e2e-extmod-a-8ecbc5-776655854d-p7qp2\" (UID: \"f4096fc6-30ab-48db-82f5-85f9847473e0\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-8ecbc5-776655854d-p7qp2"
Nov 04 03:36:07.694018 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:07.693993    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4096fc6-30ab-48db-82f5-85f9847473e0-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-8ecbc5-776655854d-p7qp2\" (UID: \"f4096fc6-30ab-48db-82f5-85f9847473e0\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-8ecbc5-776655854d-p7qp2"
Nov 04 03:36:07.720418 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:07.720371    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-8ecbc5-7df459b5c9-f2hdp"
Nov 04 03:36:07.795303 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:07.795268    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4096fc6-30ab-48db-82f5-85f9847473e0-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-8ecbc5-776655854d-p7qp2\" (UID: \"f4096fc6-30ab-48db-82f5-85f9847473e0\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-8ecbc5-776655854d-p7qp2"
Nov 04 03:36:07.795491 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:07.795370    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpjgz\" (UniqueName: \"kubernetes.io/projected/f4096fc6-30ab-48db-82f5-85f9847473e0-kube-api-access-mpjgz\") pod \"payload-processing-e2e-extmod-a-8ecbc5-776655854d-p7qp2\" (UID: \"f4096fc6-30ab-48db-82f5-85f9847473e0\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-8ecbc5-776655854d-p7qp2"
Nov 04 03:36:07.796019 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:07.795997    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4096fc6-30ab-48db-82f5-85f9847473e0-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-8ecbc5-776655854d-p7qp2\" (UID: \"f4096fc6-30ab-48db-82f5-85f9847473e0\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-8ecbc5-776655854d-p7qp2"
Nov 04 03:36:07.853568 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:07.853537    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpjgz\" (UniqueName: \"kubernetes.io/projected/f4096fc6-30ab-48db-82f5-85f9847473e0-kube-api-access-mpjgz\") pod \"payload-processing-e2e-extmod-a-8ecbc5-776655854d-p7qp2\" (UID: \"f4096fc6-30ab-48db-82f5-85f9847473e0\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-8ecbc5-776655854d-p7qp2"
Nov 04 03:36:07.903663 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:07.903628    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-8ecbc5-776655854d-p7qp2"
Nov 04 03:36:08.096775 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:08.096741    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-8ecbc5-7df459b5c9-f2hdp"]
Nov 04 03:36:08.101927 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:36:08.101900    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb263e963_52bb_4426_b6e6_e0b18f1ee810.slice/crio-2c631e235303a912235f16d94d47c630b23936ef86b268b919de382b871ca267 WatchSource:0}: Error finding container 2c631e235303a912235f16d94d47c630b23936ef86b268b919de382b871ca267: Status 404 returned error can't find the container with id 2c631e235303a912235f16d94d47c630b23936ef86b268b919de382b871ca267
Nov 04 03:36:08.341065 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:08.341030    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-8ecbc5-776655854d-p7qp2"]
Nov 04 03:36:08.346022 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:36:08.345987    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4096fc6_30ab_48db_82f5_85f9847473e0.slice/crio-44f3c80fbcf00f9177031ce07b6d3640452a491bf4d3b47ab42b40e4a47d2adb WatchSource:0}: Error finding container 44f3c80fbcf00f9177031ce07b6d3640452a491bf4d3b47ab42b40e4a47d2adb: Status 404 returned error can't find the container with id 44f3c80fbcf00f9177031ce07b6d3640452a491bf4d3b47ab42b40e4a47d2adb
Nov 04 03:36:08.785325 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:08.785287    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-8ecbc5-776655854d-p7qp2" event={"ID":"f4096fc6-30ab-48db-82f5-85f9847473e0","Type":"ContainerStarted","Data":"1214221dd74bd9b042a302adf89e3cac1c8dcd8655770aea6f5e04515de85e17"}
Nov 04 03:36:08.785896 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:08.785345    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-8ecbc5-776655854d-p7qp2" event={"ID":"f4096fc6-30ab-48db-82f5-85f9847473e0","Type":"ContainerStarted","Data":"44f3c80fbcf00f9177031ce07b6d3640452a491bf4d3b47ab42b40e4a47d2adb"}
Nov 04 03:36:08.785896 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:08.785371    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-8ecbc5-776655854d-p7qp2"
Nov 04 03:36:08.786978 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:08.786957    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-8ecbc5-7df459b5c9-f2hdp" event={"ID":"b263e963-52bb-4426-b6e6-e0b18f1ee810","Type":"ContainerStarted","Data":"8f914a65668790af43405e0226c3985efd8914101b4b46193e7ab513e6b79e26"}
Nov 04 03:36:08.787056 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:08.787003    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-8ecbc5-7df459b5c9-f2hdp" event={"ID":"b263e963-52bb-4426-b6e6-e0b18f1ee810","Type":"ContainerStarted","Data":"2c631e235303a912235f16d94d47c630b23936ef86b268b919de382b871ca267"}
Nov 04 03:36:08.787110 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:08.787046    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-8ecbc5-7df459b5c9-f2hdp"
Nov 04 03:36:08.897124 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:08.897072    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-a-8ecbc5-776655854d-p7qp2" podStartSLOduration=1.89705988 podStartE2EDuration="1.89705988s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:36:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:36:08.89529703 +0000 UTC m=+1111.471949949" watchObservedRunningTime="2026-11-04 03:36:08.89705988 +0000 UTC m=+1111.473712793"
Nov 04 03:36:09.010255 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:09.008784    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-8ecbc5-7df459b5c9-f2hdp" podStartSLOduration=2.008765712 podStartE2EDuration="2.008765712s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:36:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:36:09.005130653 +0000 UTC m=+1111.581783567" watchObservedRunningTime="2026-11-04 03:36:09.008765712 +0000 UTC m=+1111.585418630"
Nov 04 03:36:12.291911 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:12.291875    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-5d4032-854b9cd759-rcv54"]
Nov 04 03:36:12.300160 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:12.300139    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-5d4032-854b9cd759-rcv54"
Nov 04 03:36:12.349520 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:12.349486    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-5d4032-854b9cd759-rcv54"]
Nov 04 03:36:12.363603 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:12.363567    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpl7t\" (UniqueName: \"kubernetes.io/projected/0f90496f-05cc-4b88-ab4f-a7d803235b1f-kube-api-access-vpl7t\") pod \"payload-pre-processing-e2e-extmod-a-5d4032-854b9cd759-rcv54\" (UID: \"0f90496f-05cc-4b88-ab4f-a7d803235b1f\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-5d4032-854b9cd759-rcv54"
Nov 04 03:36:12.363784 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:12.363730    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f90496f-05cc-4b88-ab4f-a7d803235b1f-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-5d4032-854b9cd759-rcv54\" (UID: \"0f90496f-05cc-4b88-ab4f-a7d803235b1f\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-5d4032-854b9cd759-rcv54"
Nov 04 03:36:12.425424 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:12.403261    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-a-5d4032\""
Nov 04 03:36:12.425424 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:12.403971    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-a-5d4032-dockercfg-7qjjr\""
Nov 04 03:36:12.436276 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:12.436235    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-5d4032-6d7c8b496d-5rzsv"]
Nov 04 03:36:12.465188 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:12.465130    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpl7t\" (UniqueName: \"kubernetes.io/projected/0f90496f-05cc-4b88-ab4f-a7d803235b1f-kube-api-access-vpl7t\") pod \"payload-pre-processing-e2e-extmod-a-5d4032-854b9cd759-rcv54\" (UID: \"0f90496f-05cc-4b88-ab4f-a7d803235b1f\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-5d4032-854b9cd759-rcv54"
Nov 04 03:36:12.468269 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:12.468235    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f90496f-05cc-4b88-ab4f-a7d803235b1f-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-5d4032-854b9cd759-rcv54\" (UID: \"0f90496f-05cc-4b88-ab4f-a7d803235b1f\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-5d4032-854b9cd759-rcv54"
Nov 04 03:36:12.469456 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:12.469430    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f90496f-05cc-4b88-ab4f-a7d803235b1f-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-5d4032-854b9cd759-rcv54\" (UID: \"0f90496f-05cc-4b88-ab4f-a7d803235b1f\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-5d4032-854b9cd759-rcv54"
Nov 04 03:36:12.491431 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:12.487065    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-5d4032-6d7c8b496d-5rzsv"
Nov 04 03:36:12.496878 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:12.496231    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-5d4032-6d7c8b496d-5rzsv"]
Nov 04 03:36:12.536539 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:12.535472    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpl7t\" (UniqueName: \"kubernetes.io/projected/0f90496f-05cc-4b88-ab4f-a7d803235b1f-kube-api-access-vpl7t\") pod \"payload-pre-processing-e2e-extmod-a-5d4032-854b9cd759-rcv54\" (UID: \"0f90496f-05cc-4b88-ab4f-a7d803235b1f\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-5d4032-854b9cd759-rcv54"
Nov 04 03:36:12.570614 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:12.570193    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9qs6\" (UniqueName: \"kubernetes.io/projected/eda587fc-4491-448f-ae4a-1c8be7213bde-kube-api-access-s9qs6\") pod \"payload-processing-e2e-extmod-a-5d4032-6d7c8b496d-5rzsv\" (UID: \"eda587fc-4491-448f-ae4a-1c8be7213bde\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-5d4032-6d7c8b496d-5rzsv"
Nov 04 03:36:12.570614 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:12.570306    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eda587fc-4491-448f-ae4a-1c8be7213bde-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-5d4032-6d7c8b496d-5rzsv\" (UID: \"eda587fc-4491-448f-ae4a-1c8be7213bde\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-5d4032-6d7c8b496d-5rzsv"
Nov 04 03:36:12.619676 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:12.619643    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-5d4032-854b9cd759-rcv54"
Nov 04 03:36:12.671637 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:12.671196    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9qs6\" (UniqueName: \"kubernetes.io/projected/eda587fc-4491-448f-ae4a-1c8be7213bde-kube-api-access-s9qs6\") pod \"payload-processing-e2e-extmod-a-5d4032-6d7c8b496d-5rzsv\" (UID: \"eda587fc-4491-448f-ae4a-1c8be7213bde\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-5d4032-6d7c8b496d-5rzsv"
Nov 04 03:36:12.671637 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:12.671298    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eda587fc-4491-448f-ae4a-1c8be7213bde-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-5d4032-6d7c8b496d-5rzsv\" (UID: \"eda587fc-4491-448f-ae4a-1c8be7213bde\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-5d4032-6d7c8b496d-5rzsv"
Nov 04 03:36:12.672817 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:12.672527    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eda587fc-4491-448f-ae4a-1c8be7213bde-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-5d4032-6d7c8b496d-5rzsv\" (UID: \"eda587fc-4491-448f-ae4a-1c8be7213bde\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-5d4032-6d7c8b496d-5rzsv"
Nov 04 03:36:12.731336 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:12.731271    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9qs6\" (UniqueName: \"kubernetes.io/projected/eda587fc-4491-448f-ae4a-1c8be7213bde-kube-api-access-s9qs6\") pod \"payload-processing-e2e-extmod-a-5d4032-6d7c8b496d-5rzsv\" (UID: \"eda587fc-4491-448f-ae4a-1c8be7213bde\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-5d4032-6d7c8b496d-5rzsv"
Nov 04 03:36:12.813537 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:12.813230    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-5d4032-6d7c8b496d-5rzsv"
Nov 04 03:36:13.014424 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:13.014376    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-5d4032-854b9cd759-rcv54"]
Nov 04 03:36:13.018952 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:36:13.018922    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f90496f_05cc_4b88_ab4f_a7d803235b1f.slice/crio-6208d12929d65aaa6378665982c67bef3b0809a59613ec19d19b419316798430 WatchSource:0}: Error finding container 6208d12929d65aaa6378665982c67bef3b0809a59613ec19d19b419316798430: Status 404 returned error can't find the container with id 6208d12929d65aaa6378665982c67bef3b0809a59613ec19d19b419316798430
Nov 04 03:36:13.201124 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:13.201077    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-5d4032-6d7c8b496d-5rzsv"]
Nov 04 03:36:13.205419 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:36:13.205375    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeda587fc_4491_448f_ae4a_1c8be7213bde.slice/crio-1c1bc79249c1ceefffb17c1ddc729510781129c128dab0203a509531fb0b73bc WatchSource:0}: Error finding container 1c1bc79249c1ceefffb17c1ddc729510781129c128dab0203a509531fb0b73bc: Status 404 returned error can't find the container with id 1c1bc79249c1ceefffb17c1ddc729510781129c128dab0203a509531fb0b73bc
Nov 04 03:36:13.825952 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:13.825906    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-5d4032-6d7c8b496d-5rzsv" event={"ID":"eda587fc-4491-448f-ae4a-1c8be7213bde","Type":"ContainerStarted","Data":"156dd70e8bb480f1fb56ce107162945fa8b3fed805a5a7ae675af5cd8883c06d"}
Nov 04 03:36:13.826367 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:13.825979    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-5d4032-6d7c8b496d-5rzsv"
Nov 04 03:36:13.826367 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:13.826003    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-5d4032-6d7c8b496d-5rzsv" event={"ID":"eda587fc-4491-448f-ae4a-1c8be7213bde","Type":"ContainerStarted","Data":"1c1bc79249c1ceefffb17c1ddc729510781129c128dab0203a509531fb0b73bc"}
Nov 04 03:36:13.827773 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:13.827743    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-5d4032-854b9cd759-rcv54" event={"ID":"0f90496f-05cc-4b88-ab4f-a7d803235b1f","Type":"ContainerStarted","Data":"70e4b5b3f959ca3e0e5f5bf1557e89435b6c3fea0cf68b1043eaefab0c3e7c0e"}
Nov 04 03:36:13.827893 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:13.827791    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-5d4032-854b9cd759-rcv54" event={"ID":"0f90496f-05cc-4b88-ab4f-a7d803235b1f","Type":"ContainerStarted","Data":"6208d12929d65aaa6378665982c67bef3b0809a59613ec19d19b419316798430"}
Nov 04 03:36:13.827893 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:13.827823    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-5d4032-854b9cd759-rcv54"
Nov 04 03:36:13.943918 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:13.943866    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-a-5d4032-6d7c8b496d-5rzsv" podStartSLOduration=1.94385186 podStartE2EDuration="1.94385186s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:36:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:36:13.941078972 +0000 UTC m=+1116.517731898" watchObservedRunningTime="2026-11-04 03:36:13.94385186 +0000 UTC m=+1116.520504773"
Nov 04 03:36:14.051745 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:14.051684    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-5d4032-854b9cd759-rcv54" podStartSLOduration=2.051665168 podStartE2EDuration="2.051665168s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:36:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:36:14.05003592 +0000 UTC m=+1116.626688834" watchObservedRunningTime="2026-11-04 03:36:14.051665168 +0000 UTC m=+1116.628318082"
Nov 04 03:36:18.560214 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:36:18.560175    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:36:19.796579 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:19.796546    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-8ecbc5-776655854d-p7qp2"
Nov 04 03:36:19.797109 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:19.796628    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-8ecbc5-7df459b5c9-f2hdp"
Nov 04 03:36:24.836961 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:24.836935    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-5d4032-854b9cd759-rcv54"
Nov 04 03:36:24.837391 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:24.836997    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-5d4032-6d7c8b496d-5rzsv"
Nov 04 03:36:28.472851 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:28.472816    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd"]
Nov 04 03:36:28.476879 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:28.476849    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd"
Nov 04 03:36:28.575341 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:28.575305    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd"]
Nov 04 03:36:28.652171 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:28.652137    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/297439ad-252f-487e-a47b-3cf22cddb1b6-istiod-ca-cert\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd\" (UID: \"297439ad-252f-487e-a47b-3cf22cddb1b6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd"
Nov 04 03:36:28.652171 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:28.652175    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/297439ad-252f-487e-a47b-3cf22cddb1b6-istio-envoy\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd\" (UID: \"297439ad-252f-487e-a47b-3cf22cddb1b6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd"
Nov 04 03:36:28.652373 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:28.652219    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/297439ad-252f-487e-a47b-3cf22cddb1b6-workload-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd\" (UID: \"297439ad-252f-487e-a47b-3cf22cddb1b6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd"
Nov 04 03:36:28.652373 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:28.652337    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/297439ad-252f-487e-a47b-3cf22cddb1b6-istio-podinfo\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd\" (UID: \"297439ad-252f-487e-a47b-3cf22cddb1b6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd"
Nov 04 03:36:28.652475 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:28.652379    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/297439ad-252f-487e-a47b-3cf22cddb1b6-istio-data\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd\" (UID: \"297439ad-252f-487e-a47b-3cf22cddb1b6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd"
Nov 04 03:36:28.652510 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:28.652476    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/297439ad-252f-487e-a47b-3cf22cddb1b6-credential-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd\" (UID: \"297439ad-252f-487e-a47b-3cf22cddb1b6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd"
Nov 04 03:36:28.652545 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:28.652510    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhwxc\" (UniqueName: \"kubernetes.io/projected/297439ad-252f-487e-a47b-3cf22cddb1b6-kube-api-access-dhwxc\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd\" (UID: \"297439ad-252f-487e-a47b-3cf22cddb1b6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd"
Nov 04 03:36:28.652630 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:28.652607    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/297439ad-252f-487e-a47b-3cf22cddb1b6-workload-certs\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd\" (UID: \"297439ad-252f-487e-a47b-3cf22cddb1b6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd"
Nov 04 03:36:28.652673 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:28.652646    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/297439ad-252f-487e-a47b-3cf22cddb1b6-istio-token\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd\" (UID: \"297439ad-252f-487e-a47b-3cf22cddb1b6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd"
Nov 04 03:36:28.753585 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:28.753503    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhwxc\" (UniqueName: \"kubernetes.io/projected/297439ad-252f-487e-a47b-3cf22cddb1b6-kube-api-access-dhwxc\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd\" (UID: \"297439ad-252f-487e-a47b-3cf22cddb1b6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd"
Nov 04 03:36:28.753585 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:28.753551    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/297439ad-252f-487e-a47b-3cf22cddb1b6-workload-certs\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd\" (UID: \"297439ad-252f-487e-a47b-3cf22cddb1b6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd"
Nov 04 03:36:28.753814 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:28.753656    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/297439ad-252f-487e-a47b-3cf22cddb1b6-istio-token\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd\" (UID: \"297439ad-252f-487e-a47b-3cf22cddb1b6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd"
Nov 04 03:36:28.753814 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:28.753719    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/297439ad-252f-487e-a47b-3cf22cddb1b6-istiod-ca-cert\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd\" (UID: \"297439ad-252f-487e-a47b-3cf22cddb1b6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd"
Nov 04 03:36:28.753814 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:28.753742    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/297439ad-252f-487e-a47b-3cf22cddb1b6-istio-envoy\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd\" (UID: \"297439ad-252f-487e-a47b-3cf22cddb1b6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd"
Nov 04 03:36:28.753814 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:28.753782    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/297439ad-252f-487e-a47b-3cf22cddb1b6-workload-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd\" (UID: \"297439ad-252f-487e-a47b-3cf22cddb1b6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd"
Nov 04 03:36:28.754084 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:28.753836    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/297439ad-252f-487e-a47b-3cf22cddb1b6-istio-podinfo\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd\" (UID: \"297439ad-252f-487e-a47b-3cf22cddb1b6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd"
Nov 04 03:36:28.754084 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:28.753858    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/297439ad-252f-487e-a47b-3cf22cddb1b6-istio-data\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd\" (UID: \"297439ad-252f-487e-a47b-3cf22cddb1b6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd"
Nov 04 03:36:28.754084 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:28.753873    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/297439ad-252f-487e-a47b-3cf22cddb1b6-workload-certs\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd\" (UID: \"297439ad-252f-487e-a47b-3cf22cddb1b6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd"
Nov 04 03:36:28.754084 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:28.753939    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/297439ad-252f-487e-a47b-3cf22cddb1b6-credential-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd\" (UID: \"297439ad-252f-487e-a47b-3cf22cddb1b6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd"
Nov 04 03:36:28.754267 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:28.754140    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/297439ad-252f-487e-a47b-3cf22cddb1b6-workload-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd\" (UID: \"297439ad-252f-487e-a47b-3cf22cddb1b6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd"
Nov 04 03:36:28.754267 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:28.754244    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/297439ad-252f-487e-a47b-3cf22cddb1b6-istio-data\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd\" (UID: \"297439ad-252f-487e-a47b-3cf22cddb1b6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd"
Nov 04 03:36:28.754335 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:28.754305    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/297439ad-252f-487e-a47b-3cf22cddb1b6-credential-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd\" (UID: \"297439ad-252f-487e-a47b-3cf22cddb1b6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd"
Nov 04 03:36:28.754533 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:28.754511    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/297439ad-252f-487e-a47b-3cf22cddb1b6-istiod-ca-cert\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd\" (UID: \"297439ad-252f-487e-a47b-3cf22cddb1b6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd"
Nov 04 03:36:28.756068 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:28.756045    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/297439ad-252f-487e-a47b-3cf22cddb1b6-istio-envoy\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd\" (UID: \"297439ad-252f-487e-a47b-3cf22cddb1b6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd"
Nov 04 03:36:28.756360 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:28.756340    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/297439ad-252f-487e-a47b-3cf22cddb1b6-istio-podinfo\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd\" (UID: \"297439ad-252f-487e-a47b-3cf22cddb1b6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd"
Nov 04 03:36:28.811884 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:28.811851    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/297439ad-252f-487e-a47b-3cf22cddb1b6-istio-token\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd\" (UID: \"297439ad-252f-487e-a47b-3cf22cddb1b6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd"
Nov 04 03:36:28.812005 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:28.811941    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhwxc\" (UniqueName: \"kubernetes.io/projected/297439ad-252f-487e-a47b-3cf22cddb1b6-kube-api-access-dhwxc\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd\" (UID: \"297439ad-252f-487e-a47b-3cf22cddb1b6\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd"
Nov 04 03:36:29.091623 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:29.091543    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd"
Nov 04 03:36:29.520690 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:29.520654    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd"]
Nov 04 03:36:29.526040 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:36:29.526002    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod297439ad_252f_487e_a47b_3cf22cddb1b6.slice/crio-6d5047a59560311ff14b0da2d5d54b5b35c43d3833425b78e67bdf4e7799a7e3 WatchSource:0}: Error finding container 6d5047a59560311ff14b0da2d5d54b5b35c43d3833425b78e67bdf4e7799a7e3: Status 404 returned error can't find the container with id 6d5047a59560311ff14b0da2d5d54b5b35c43d3833425b78e67bdf4e7799a7e3
Nov 04 03:36:29.528849 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:29.528807    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:36:29.528942 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:29.528898    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:36:29.528983 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:29.528941    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:36:29.933654 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:29.933620    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd" event={"ID":"297439ad-252f-487e-a47b-3cf22cddb1b6","Type":"ContainerStarted","Data":"638f27e833bc1f91dd51e7442824a9efd31da0b2d25326765ffc7d0934a8cf81"}
Nov 04 03:36:29.933834 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:29.933683    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd" event={"ID":"297439ad-252f-487e-a47b-3cf22cddb1b6","Type":"ContainerStarted","Data":"6d5047a59560311ff14b0da2d5d54b5b35c43d3833425b78e67bdf4e7799a7e3"}
Nov 04 03:36:30.051685 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:30.051625    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd" podStartSLOduration=2.051608843 podStartE2EDuration="2.051608843s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:36:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:36:30.048782829 +0000 UTC m=+1132.625435769" watchObservedRunningTime="2026-11-04 03:36:30.051608843 +0000 UTC m=+1132.628261756"
Nov 04 03:36:30.092311 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:30.092276    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd"
Nov 04 03:36:30.094265 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:30.094228    5609 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 04 03:36:30.094430 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:30.094297    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd" podUID="297439ad-252f-487e-a47b-3cf22cddb1b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 04 03:36:31.105424 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:31.096821    5609 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 04 03:36:31.105424 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:31.096895    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd" podUID="297439ad-252f-487e-a47b-3cf22cddb1b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 04 03:36:31.809425 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:31.800533    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-8ecbc5-59968fdc4b-lsf4c"]
Nov 04 03:36:31.814449 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:31.813347    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-8ecbc5-59968fdc4b-lsf4c"
Nov 04 03:36:31.846499 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:31.846470    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-8ecbc5-59968fdc4b-lsf4c"]
Nov 04 03:36:31.898425 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:31.892677    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-a-8ecbc5-serving-cert\""
Nov 04 03:36:31.898425 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:31.893464    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-cpkkm\""
Nov 04 03:36:31.898425 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:31.893585    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Nov 04 03:36:31.914452 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:31.905050    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7610c80-027e-4494-8f93-2d275c06c796-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-8ecbc5-59968fdc4b-lsf4c\" (UID: \"d7610c80-027e-4494-8f93-2d275c06c796\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-8ecbc5-59968fdc4b-lsf4c"
Nov 04 03:36:31.914452 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:31.905112    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7610c80-027e-4494-8f93-2d275c06c796-maas-api-tls\") pod \"maas-api-e2e-extmod-a-8ecbc5-59968fdc4b-lsf4c\" (UID: \"d7610c80-027e-4494-8f93-2d275c06c796\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-8ecbc5-59968fdc4b-lsf4c"
Nov 04 03:36:31.914452 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:31.905241    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sg76\" (UniqueName: \"kubernetes.io/projected/d7610c80-027e-4494-8f93-2d275c06c796-kube-api-access-9sg76\") pod \"maas-api-e2e-extmod-a-8ecbc5-59968fdc4b-lsf4c\" (UID: \"d7610c80-027e-4494-8f93-2d275c06c796\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-8ecbc5-59968fdc4b-lsf4c"
Nov 04 03:36:31.914452 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:31.905299    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7610c80-027e-4494-8f93-2d275c06c796-metrics-certs\") pod \"maas-api-e2e-extmod-a-8ecbc5-59968fdc4b-lsf4c\" (UID: \"d7610c80-027e-4494-8f93-2d275c06c796\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-8ecbc5-59968fdc4b-lsf4c"
Nov 04 03:36:32.015443 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:32.015103    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7610c80-027e-4494-8f93-2d275c06c796-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-8ecbc5-59968fdc4b-lsf4c\" (UID: \"d7610c80-027e-4494-8f93-2d275c06c796\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-8ecbc5-59968fdc4b-lsf4c"
Nov 04 03:36:32.015443 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:32.015172    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7610c80-027e-4494-8f93-2d275c06c796-maas-api-tls\") pod \"maas-api-e2e-extmod-a-8ecbc5-59968fdc4b-lsf4c\" (UID: \"d7610c80-027e-4494-8f93-2d275c06c796\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-8ecbc5-59968fdc4b-lsf4c"
Nov 04 03:36:32.015443 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:32.015305    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9sg76\" (UniqueName: \"kubernetes.io/projected/d7610c80-027e-4494-8f93-2d275c06c796-kube-api-access-9sg76\") pod \"maas-api-e2e-extmod-a-8ecbc5-59968fdc4b-lsf4c\" (UID: \"d7610c80-027e-4494-8f93-2d275c06c796\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-8ecbc5-59968fdc4b-lsf4c"
Nov 04 03:36:32.015443 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:32.015367    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7610c80-027e-4494-8f93-2d275c06c796-metrics-certs\") pod \"maas-api-e2e-extmod-a-8ecbc5-59968fdc4b-lsf4c\" (UID: \"d7610c80-027e-4494-8f93-2d275c06c796\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-8ecbc5-59968fdc4b-lsf4c"
Nov 04 03:36:32.016058 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:36:32.015861    5609 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-a-8ecbc5-serving-cert: secret "maas-api-e2e-extmod-a-8ecbc5-serving-cert" not found
Nov 04 03:36:32.016271 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:36:32.016254    5609 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7610c80-027e-4494-8f93-2d275c06c796-maas-api-tls podName:d7610c80-027e-4494-8f93-2d275c06c796 nodeName:}" failed. No retries permitted until 2026-11-04 03:36:32.516229702 +0000 UTC m=+1135.092882610 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d7610c80-027e-4494-8f93-2d275c06c796-maas-api-tls") pod "maas-api-e2e-extmod-a-8ecbc5-59968fdc4b-lsf4c" (UID: "d7610c80-027e-4494-8f93-2d275c06c796") : secret "maas-api-e2e-extmod-a-8ecbc5-serving-cert" not found
Nov 04 03:36:32.016384 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:32.016284    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7610c80-027e-4494-8f93-2d275c06c796-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-8ecbc5-59968fdc4b-lsf4c\" (UID: \"d7610c80-027e-4494-8f93-2d275c06c796\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-8ecbc5-59968fdc4b-lsf4c"
Nov 04 03:36:32.018881 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:32.018852    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7610c80-027e-4494-8f93-2d275c06c796-metrics-certs\") pod \"maas-api-e2e-extmod-a-8ecbc5-59968fdc4b-lsf4c\" (UID: \"d7610c80-027e-4494-8f93-2d275c06c796\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-8ecbc5-59968fdc4b-lsf4c"
Nov 04 03:36:32.079158 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:32.079081    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sg76\" (UniqueName: \"kubernetes.io/projected/d7610c80-027e-4494-8f93-2d275c06c796-kube-api-access-9sg76\") pod \"maas-api-e2e-extmod-a-8ecbc5-59968fdc4b-lsf4c\" (UID: \"d7610c80-027e-4494-8f93-2d275c06c796\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-8ecbc5-59968fdc4b-lsf4c"
Nov 04 03:36:32.102824 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:32.102790    5609 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 04 03:36:32.103097 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:32.103068    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd" podUID="297439ad-252f-487e-a47b-3cf22cddb1b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 04 03:36:32.525118 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:32.525080    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7610c80-027e-4494-8f93-2d275c06c796-maas-api-tls\") pod \"maas-api-e2e-extmod-a-8ecbc5-59968fdc4b-lsf4c\" (UID: \"d7610c80-027e-4494-8f93-2d275c06c796\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-8ecbc5-59968fdc4b-lsf4c"
Nov 04 03:36:32.527696 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:32.527667    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7610c80-027e-4494-8f93-2d275c06c796-maas-api-tls\") pod \"maas-api-e2e-extmod-a-8ecbc5-59968fdc4b-lsf4c\" (UID: \"d7610c80-027e-4494-8f93-2d275c06c796\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-8ecbc5-59968fdc4b-lsf4c"
Nov 04 03:36:32.731780 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:32.731731    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-8ecbc5-59968fdc4b-lsf4c"
Nov 04 03:36:33.098309 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:33.097782    5609 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 04 03:36:33.098309 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:33.097844    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd" podUID="297439ad-252f-487e-a47b-3cf22cddb1b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 04 03:36:33.169469 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:33.160262    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-8ecbc5-59968fdc4b-lsf4c"]
Nov 04 03:36:33.171950 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:36:33.171917    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7610c80_027e_4494_8f93_2d275c06c796.slice/crio-6b26a9aa44d61f36dc303864ff67bb2c286cb2ccea776473b8987caa80194ebf WatchSource:0}: Error finding container 6b26a9aa44d61f36dc303864ff67bb2c286cb2ccea776473b8987caa80194ebf: Status 404 returned error can't find the container with id 6b26a9aa44d61f36dc303864ff67bb2c286cb2ccea776473b8987caa80194ebf
Nov 04 03:36:33.967702 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:33.967662    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-8ecbc5-59968fdc4b-lsf4c" event={"ID":"d7610c80-027e-4494-8f93-2d275c06c796","Type":"ContainerStarted","Data":"6b26a9aa44d61f36dc303864ff67bb2c286cb2ccea776473b8987caa80194ebf"}
Nov 04 03:36:34.096825 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:34.096782    5609 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 04 03:36:34.097102 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:34.097073    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd" podUID="297439ad-252f-487e-a47b-3cf22cddb1b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 04 03:36:35.102221 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:35.101798    5609 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 04 03:36:35.102221 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:35.101857    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd" podUID="297439ad-252f-487e-a47b-3cf22cddb1b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 04 03:36:35.990364 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:35.990328    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd"]
Nov 04 03:36:35.990636 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:35.990602    5609 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd" podUID="fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a" containerName="istio-proxy" containerID="cri-o://4baa554d96d0535db710ba2fd7ea5ac560841a0877a078c08e40bdfe5f5e6c5d" gracePeriod=30
Nov 04 03:36:36.008531 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:36.008492    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-8ecbc5-59968fdc4b-lsf4c" event={"ID":"d7610c80-027e-4494-8f93-2d275c06c796","Type":"ContainerStarted","Data":"00fd37b445a5310bdfc77be17926dfa416526a53a359b391d838ed56e2629449"}
Nov 04 03:36:36.008705 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:36.008566    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-8ecbc5-59968fdc4b-lsf4c"
Nov 04 03:36:36.092781 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:36.092747    5609 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 04 03:36:36.092951 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:36.092815    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd" podUID="297439ad-252f-487e-a47b-3cf22cddb1b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 04 03:36:36.233673 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:36.233608    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-8ecbc5-59968fdc4b-lsf4c" podStartSLOduration=2.523492764 podStartE2EDuration="5.233587936s" totalImagesPullingTime="2.710095172s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:36:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:36:36.230994824 +0000 UTC m=+1138.807647760" watchObservedRunningTime="2026-11-04 03:36:36.233587936 +0000 UTC m=+1138.810240850"
Nov 04 03:36:37.097006 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:37.096974    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd"
Nov 04 03:36:37.097822 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:37.097634    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd"
Nov 04 03:36:37.098583 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:37.098561    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd"
Nov 04 03:36:41.247184 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:41.247157    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd"
Nov 04 03:36:41.343981 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:41.343907    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-istio-envoy\") pod \"fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a\" (UID: \"fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a\") "
Nov 04 03:36:41.343981 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:41.343947    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-kube-api-access-q7fvv\" (UniqueName: \"kubernetes.io/projected/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-kube-api-access-q7fvv\") pod \"fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a\" (UID: \"fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a\") "
Nov 04 03:36:41.343981 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:41.343972    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-istio-token\" (UniqueName: \"kubernetes.io/projected/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-istio-token\") pod \"fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a\" (UID: \"fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a\") "
Nov 04 03:36:41.344282 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:41.343992    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-credential-socket\") pod \"fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a\" (UID: \"fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a\") "
Nov 04 03:36:41.344282 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:41.344047    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-workload-certs\") pod \"fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a\" (UID: \"fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a\") "
Nov 04 03:36:41.344282 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:41.344079    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-istiod-ca-cert\") pod \"fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a\" (UID: \"fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a\") "
Nov 04 03:36:41.344282 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:41.344128    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-istio-data\") pod \"fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a\" (UID: \"fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a\") "
Nov 04 03:36:41.344282 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:41.344191    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-istio-podinfo\") pod \"fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a\" (UID: \"fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a\") "
Nov 04 03:36:41.344282 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:41.344274    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-workload-socket\") pod \"fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a\" (UID: \"fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a\") "
Nov 04 03:36:41.344639 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:41.344345    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-credential-socket" pod "fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a" (UID: "fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:36:41.344639 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:41.344511    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-istio-data" pod "fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a" (UID: "fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:36:41.344743 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:41.344722    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-workload-socket" pod "fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a" (UID: "fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:36:41.344808 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:41.344742    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-workload-certs" pod "fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a" (UID: "fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:36:41.344969 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:41.344949    5609 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-istio-data\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:36:41.345043 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:41.344974    5609 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-workload-socket\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:36:41.345043 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:41.344993    5609 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-credential-socket\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:36:41.345043 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:41.345007    5609 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-workload-certs\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:36:41.345043 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:41.345031    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-istiod-ca-cert" pod "fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a" (UID: "fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:36:41.346312 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:41.346291    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-istio-envoy" pod "fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a" (UID: "fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:36:41.346312 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:41.346303    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-kube-api-access-q7fvv" pod "fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a" (UID: "fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a"). InnerVolumeSpecName "kube-api-access-q7fvv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:36:41.346517 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:41.346306    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-istio-token" pod "fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a" (UID: "fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:36:41.346517 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:41.346346    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-istio-podinfo" pod "fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a" (UID: "fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:36:41.446186 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:41.446145    5609 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-istio-envoy\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:36:41.446186 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:41.446183    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q7fvv\" (UniqueName: \"kubernetes.io/projected/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-kube-api-access-q7fvv\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:36:41.446379 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:41.446195    5609 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-istio-token\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:36:41.446379 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:41.446205    5609 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-istiod-ca-cert\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:36:41.446379 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:41.446219    5609 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a-istio-podinfo\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:36:41.853514 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:41.853478    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-5d4032-7589d67fb-xml6f"]
Nov 04 03:36:41.854200 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:41.854179    5609 cpu_manager.go:414] "containerMap: removing container" podUID="fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a" containerName="istio-proxy"
Nov 04 03:36:41.854200 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:41.854199    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a" containerName="istio-proxy"
Nov 04 03:36:41.854396 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:41.854350    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a" containerName="istio-proxy"
Nov 04 03:36:41.858183 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:41.858163    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5d4032-7589d67fb-xml6f"
Nov 04 03:36:41.909680 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:41.909593    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-a-5d4032-serving-cert\""
Nov 04 03:36:41.914866 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:41.914841    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-5d4032-7589d67fb-xml6f"]
Nov 04 03:36:41.950873 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:41.950842    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81dc8aa9-657a-4504-b306-177c32c1774e-maas-api-tls\") pod \"maas-api-e2e-extmod-a-5d4032-7589d67fb-xml6f\" (UID: \"81dc8aa9-657a-4504-b306-177c32c1774e\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5d4032-7589d67fb-xml6f"
Nov 04 03:36:41.951039 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:41.950990    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81dc8aa9-657a-4504-b306-177c32c1774e-metrics-certs\") pod \"maas-api-e2e-extmod-a-5d4032-7589d67fb-xml6f\" (UID: \"81dc8aa9-657a-4504-b306-177c32c1774e\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5d4032-7589d67fb-xml6f"
Nov 04 03:36:41.951039 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:41.951022    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkrz6\" (UniqueName: \"kubernetes.io/projected/81dc8aa9-657a-4504-b306-177c32c1774e-kube-api-access-bkrz6\") pod \"maas-api-e2e-extmod-a-5d4032-7589d67fb-xml6f\" (UID: \"81dc8aa9-657a-4504-b306-177c32c1774e\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5d4032-7589d67fb-xml6f"
Nov 04 03:36:41.951171 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:41.951055    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81dc8aa9-657a-4504-b306-177c32c1774e-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-5d4032-7589d67fb-xml6f\" (UID: \"81dc8aa9-657a-4504-b306-177c32c1774e\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5d4032-7589d67fb-xml6f"
Nov 04 03:36:42.020434 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:42.020376    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-8ecbc5-59968fdc4b-lsf4c"
Nov 04 03:36:42.051708 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:42.051673    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81dc8aa9-657a-4504-b306-177c32c1774e-metrics-certs\") pod \"maas-api-e2e-extmod-a-5d4032-7589d67fb-xml6f\" (UID: \"81dc8aa9-657a-4504-b306-177c32c1774e\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5d4032-7589d67fb-xml6f"
Nov 04 03:36:42.051858 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:42.051718    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkrz6\" (UniqueName: \"kubernetes.io/projected/81dc8aa9-657a-4504-b306-177c32c1774e-kube-api-access-bkrz6\") pod \"maas-api-e2e-extmod-a-5d4032-7589d67fb-xml6f\" (UID: \"81dc8aa9-657a-4504-b306-177c32c1774e\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5d4032-7589d67fb-xml6f"
Nov 04 03:36:42.051858 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:42.051758    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81dc8aa9-657a-4504-b306-177c32c1774e-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-5d4032-7589d67fb-xml6f\" (UID: \"81dc8aa9-657a-4504-b306-177c32c1774e\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5d4032-7589d67fb-xml6f"
Nov 04 03:36:42.051858 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:42.051826    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81dc8aa9-657a-4504-b306-177c32c1774e-maas-api-tls\") pod \"maas-api-e2e-extmod-a-5d4032-7589d67fb-xml6f\" (UID: \"81dc8aa9-657a-4504-b306-177c32c1774e\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5d4032-7589d67fb-xml6f"
Nov 04 03:36:42.052090 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:36:42.052068    5609 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-a-5d4032-serving-cert: secret "maas-api-e2e-extmod-a-5d4032-serving-cert" not found
Nov 04 03:36:42.052156 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:36:42.052137    5609 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81dc8aa9-657a-4504-b306-177c32c1774e-maas-api-tls podName:81dc8aa9-657a-4504-b306-177c32c1774e nodeName:}" failed. No retries permitted until 2026-11-04 03:36:42.552120517 +0000 UTC m=+1145.128773411 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/81dc8aa9-657a-4504-b306-177c32c1774e-maas-api-tls") pod "maas-api-e2e-extmod-a-5d4032-7589d67fb-xml6f" (UID: "81dc8aa9-657a-4504-b306-177c32c1774e") : secret "maas-api-e2e-extmod-a-5d4032-serving-cert" not found
Nov 04 03:36:42.052227 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:42.052148    5609 generic.go:356] "Generic (PLEG): container finished" podID="fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a" containerID="4baa554d96d0535db710ba2fd7ea5ac560841a0877a078c08e40bdfe5f5e6c5d" exitCode=0
Nov 04 03:36:42.052282 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:42.052247    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd"
Nov 04 03:36:42.052337 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:42.052286    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd" event={"ID":"fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a","Type":"ContainerDied","Data":"4baa554d96d0535db710ba2fd7ea5ac560841a0877a078c08e40bdfe5f5e6c5d"}
Nov 04 03:36:42.052396 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:42.052341    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd" event={"ID":"fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a","Type":"ContainerDied","Data":"efbebbb11fa0d5a099d1caffe18753900d9b828087208056e92884a629e0eeb8"}
Nov 04 03:36:42.052396 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:42.052370    5609 scope.go:122] "RemoveContainer" containerID="4baa554d96d0535db710ba2fd7ea5ac560841a0877a078c08e40bdfe5f5e6c5d"
Nov 04 03:36:42.052651 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:42.052628    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81dc8aa9-657a-4504-b306-177c32c1774e-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-5d4032-7589d67fb-xml6f\" (UID: \"81dc8aa9-657a-4504-b306-177c32c1774e\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5d4032-7589d67fb-xml6f"
Nov 04 03:36:42.054558 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:42.054531    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81dc8aa9-657a-4504-b306-177c32c1774e-metrics-certs\") pod \"maas-api-e2e-extmod-a-5d4032-7589d67fb-xml6f\" (UID: \"81dc8aa9-657a-4504-b306-177c32c1774e\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5d4032-7589d67fb-xml6f"
Nov 04 03:36:42.070446 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:42.070425    5609 scope.go:122] "RemoveContainer" containerID="4baa554d96d0535db710ba2fd7ea5ac560841a0877a078c08e40bdfe5f5e6c5d"
Nov 04 03:36:42.070721 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:36:42.070696    5609 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4baa554d96d0535db710ba2fd7ea5ac560841a0877a078c08e40bdfe5f5e6c5d\": container with ID starting with 4baa554d96d0535db710ba2fd7ea5ac560841a0877a078c08e40bdfe5f5e6c5d not found: ID does not exist" containerID="4baa554d96d0535db710ba2fd7ea5ac560841a0877a078c08e40bdfe5f5e6c5d"
Nov 04 03:36:42.070798 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:42.070726    5609 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4baa554d96d0535db710ba2fd7ea5ac560841a0877a078c08e40bdfe5f5e6c5d"} err="failed to get container status \"4baa554d96d0535db710ba2fd7ea5ac560841a0877a078c08e40bdfe5f5e6c5d\": rpc error: code = NotFound desc = could not find container \"4baa554d96d0535db710ba2fd7ea5ac560841a0877a078c08e40bdfe5f5e6c5d\": container with ID starting with 4baa554d96d0535db710ba2fd7ea5ac560841a0877a078c08e40bdfe5f5e6c5d not found: ID does not exist"
Nov 04 03:36:42.120249 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:42.120140    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkrz6\" (UniqueName: \"kubernetes.io/projected/81dc8aa9-657a-4504-b306-177c32c1774e-kube-api-access-bkrz6\") pod \"maas-api-e2e-extmod-a-5d4032-7589d67fb-xml6f\" (UID: \"81dc8aa9-657a-4504-b306-177c32c1774e\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5d4032-7589d67fb-xml6f"
Nov 04 03:36:42.242920 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:42.242874    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd"]
Nov 04 03:36:42.247626 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:42.247601    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-2tbrd"]
Nov 04 03:36:42.559216 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:42.559185    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81dc8aa9-657a-4504-b306-177c32c1774e-maas-api-tls\") pod \"maas-api-e2e-extmod-a-5d4032-7589d67fb-xml6f\" (UID: \"81dc8aa9-657a-4504-b306-177c32c1774e\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5d4032-7589d67fb-xml6f"
Nov 04 03:36:42.561686 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:42.561664    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81dc8aa9-657a-4504-b306-177c32c1774e-maas-api-tls\") pod \"maas-api-e2e-extmod-a-5d4032-7589d67fb-xml6f\" (UID: \"81dc8aa9-657a-4504-b306-177c32c1774e\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5d4032-7589d67fb-xml6f"
Nov 04 03:36:42.771908 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:42.771871    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5d4032-7589d67fb-xml6f"
Nov 04 03:36:43.147795 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:43.147717    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-5d4032-7589d67fb-xml6f"]
Nov 04 03:36:43.152203 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:36:43.152177    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81dc8aa9_657a_4504_b306_177c32c1774e.slice/crio-e21cfe7779a1019271449db7a28deea889bf63a18a3f47ca7531b51058fc27e0 WatchSource:0}: Error finding container e21cfe7779a1019271449db7a28deea889bf63a18a3f47ca7531b51058fc27e0: Status 404 returned error can't find the container with id e21cfe7779a1019271449db7a28deea889bf63a18a3f47ca7531b51058fc27e0
Nov 04 03:36:43.535148 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:43.535119    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a" path="/var/lib/kubelet/pods/fd9f1f55-d4c4-42b4-8cc3-41a2f87a3a8a/volumes"
Nov 04 03:36:44.068171 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:44.068132    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5d4032-7589d67fb-xml6f" event={"ID":"81dc8aa9-657a-4504-b306-177c32c1774e","Type":"ContainerStarted","Data":"e21cfe7779a1019271449db7a28deea889bf63a18a3f47ca7531b51058fc27e0"}
Nov 04 03:36:45.080119 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:45.080079    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5d4032-7589d67fb-xml6f" event={"ID":"81dc8aa9-657a-4504-b306-177c32c1774e","Type":"ContainerStarted","Data":"e8dea0fcbb40175549a31928fd95702332def5c61ee9efa9741d64beb7bc250e"}
Nov 04 03:36:45.080590 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:45.080163    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5d4032-7589d67fb-xml6f"
Nov 04 03:36:45.197915 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:45.197861    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5d4032-7589d67fb-xml6f" podStartSLOduration=3.242426122 podStartE2EDuration="4.19784731s" totalImagesPullingTime="955.421188ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:36:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:36:45.195442589 +0000 UTC m=+1147.772095503" watchObservedRunningTime="2026-11-04 03:36:45.19784731 +0000 UTC m=+1147.774500259"
Nov 04 03:36:48.560165 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:36:48.560118    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:36:51.092512 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:36:51.092480    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5d4032-7589d67fb-xml6f"
Nov 04 03:37:15.297660 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:15.297617    5609 generic.go:356] "Generic (PLEG): container finished" podID="1f315bbc-cddf-41ee-b044-92914a203e58" containerID="cade2663d5901a6475aae1eb80d2a270fcca95a241dd92b627656ff6db176ea7" exitCode=0
Nov 04 03:37:15.298187 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:15.297675    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk" event={"ID":"1f315bbc-cddf-41ee-b044-92914a203e58","Type":"ContainerDied","Data":"cade2663d5901a6475aae1eb80d2a270fcca95a241dd92b627656ff6db176ea7"}
Nov 04 03:37:15.298187 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:15.298159    5609 scope.go:122] "RemoveContainer" containerID="cade2663d5901a6475aae1eb80d2a270fcca95a241dd92b627656ff6db176ea7"
Nov 04 03:37:15.299912 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:15.299875    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:37:15.300010 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:15.299961    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:37:15.300048 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:15.300009    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:37:16.398426 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:16.394853    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk" event={"ID":"1f315bbc-cddf-41ee-b044-92914a203e58","Type":"ContainerStarted","Data":"b6a924e6e695a68b9cc923df5fef390ae54d85acfaed4426fcaa8e280d9d7149"}
Nov 04 03:37:17.338864 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:17.338829    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk"
Nov 04 03:37:17.339225 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:17.339200    5609 patch_prober.go:29] interesting pod/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 04 03:37:17.339436 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:17.339379    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk" podUID="1f315bbc-cddf-41ee-b044-92914a203e58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 04 03:37:18.335166 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:18.335118    5609 patch_prober.go:29] interesting pod/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 04 03:37:18.335710 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:18.335195    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk" podUID="1f315bbc-cddf-41ee-b044-92914a203e58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 04 03:37:18.563073 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:37:18.563037    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:37:19.334910 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:19.334863    5609 patch_prober.go:29] interesting pod/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 04 03:37:19.335094 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:19.334931    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk" podUID="1f315bbc-cddf-41ee-b044-92914a203e58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 04 03:37:20.334885 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:20.334845    5609 patch_prober.go:29] interesting pod/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 04 03:37:20.335602 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:20.335566    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk" podUID="1f315bbc-cddf-41ee-b044-92914a203e58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 04 03:37:21.339266 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:21.338780    5609 patch_prober.go:29] interesting pod/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 04 03:37:21.339266 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:21.338852    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk" podUID="1f315bbc-cddf-41ee-b044-92914a203e58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 04 03:37:22.339376 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:22.339347    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk"
Nov 04 03:37:22.339796 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:22.339659    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk"
Nov 04 03:37:22.340342 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:22.340325    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk"
Nov 04 03:37:22.635807 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:22.635737    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j"]
Nov 04 03:37:22.641314 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:22.641270    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j"
Nov 04 03:37:22.691309 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:22.691138    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-b-1bf4bf-openshift-default-dockercfg-2g8ff\""
Nov 04 03:37:22.727935 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:22.727899    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45m98\" (UniqueName: \"kubernetes.io/projected/29087b39-1965-453c-9d19-61a4df9b0156-kube-api-access-45m98\") pod \"e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j\" (UID: \"29087b39-1965-453c-9d19-61a4df9b0156\") " pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j"
Nov 04 03:37:22.728096 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:22.728014    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29087b39-1965-453c-9d19-61a4df9b0156-istio-envoy\") pod \"e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j\" (UID: \"29087b39-1965-453c-9d19-61a4df9b0156\") " pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j"
Nov 04 03:37:22.728096 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:22.728059    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29087b39-1965-453c-9d19-61a4df9b0156-credential-socket\") pod \"e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j\" (UID: \"29087b39-1965-453c-9d19-61a4df9b0156\") " pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j"
Nov 04 03:37:22.728096 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:22.728085    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29087b39-1965-453c-9d19-61a4df9b0156-istiod-ca-cert\") pod \"e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j\" (UID: \"29087b39-1965-453c-9d19-61a4df9b0156\") " pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j"
Nov 04 03:37:22.728228 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:22.728127    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29087b39-1965-453c-9d19-61a4df9b0156-istio-token\") pod \"e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j\" (UID: \"29087b39-1965-453c-9d19-61a4df9b0156\") " pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j"
Nov 04 03:37:22.728317 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:22.728293    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29087b39-1965-453c-9d19-61a4df9b0156-istio-data\") pod \"e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j\" (UID: \"29087b39-1965-453c-9d19-61a4df9b0156\") " pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j"
Nov 04 03:37:22.728484 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:22.728335    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29087b39-1965-453c-9d19-61a4df9b0156-istio-podinfo\") pod \"e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j\" (UID: \"29087b39-1965-453c-9d19-61a4df9b0156\") " pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j"
Nov 04 03:37:22.728484 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:22.728363    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29087b39-1965-453c-9d19-61a4df9b0156-workload-certs\") pod \"e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j\" (UID: \"29087b39-1965-453c-9d19-61a4df9b0156\") " pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j"
Nov 04 03:37:22.728600 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:22.728480    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29087b39-1965-453c-9d19-61a4df9b0156-workload-socket\") pod \"e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j\" (UID: \"29087b39-1965-453c-9d19-61a4df9b0156\") " pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j"
Nov 04 03:37:22.733945 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:22.733903    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j"]
Nov 04 03:37:22.832422 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:22.830091    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29087b39-1965-453c-9d19-61a4df9b0156-credential-socket\") pod \"e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j\" (UID: \"29087b39-1965-453c-9d19-61a4df9b0156\") " pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j"
Nov 04 03:37:22.832422 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:22.830151    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29087b39-1965-453c-9d19-61a4df9b0156-istiod-ca-cert\") pod \"e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j\" (UID: \"29087b39-1965-453c-9d19-61a4df9b0156\") " pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j"
Nov 04 03:37:22.832422 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:22.830212    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29087b39-1965-453c-9d19-61a4df9b0156-istio-token\") pod \"e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j\" (UID: \"29087b39-1965-453c-9d19-61a4df9b0156\") " pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j"
Nov 04 03:37:22.832422 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:22.830560    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29087b39-1965-453c-9d19-61a4df9b0156-credential-socket\") pod \"e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j\" (UID: \"29087b39-1965-453c-9d19-61a4df9b0156\") " pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j"
Nov 04 03:37:22.832422 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:22.830603    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29087b39-1965-453c-9d19-61a4df9b0156-istio-data\") pod \"e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j\" (UID: \"29087b39-1965-453c-9d19-61a4df9b0156\") " pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j"
Nov 04 03:37:22.832422 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:22.830879    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29087b39-1965-453c-9d19-61a4df9b0156-istio-data\") pod \"e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j\" (UID: \"29087b39-1965-453c-9d19-61a4df9b0156\") " pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j"
Nov 04 03:37:22.832422 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:22.830910    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29087b39-1965-453c-9d19-61a4df9b0156-istiod-ca-cert\") pod \"e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j\" (UID: \"29087b39-1965-453c-9d19-61a4df9b0156\") " pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j"
Nov 04 03:37:22.832422 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:22.830962    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29087b39-1965-453c-9d19-61a4df9b0156-istio-podinfo\") pod \"e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j\" (UID: \"29087b39-1965-453c-9d19-61a4df9b0156\") " pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j"
Nov 04 03:37:22.832422 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:22.830993    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29087b39-1965-453c-9d19-61a4df9b0156-workload-certs\") pod \"e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j\" (UID: \"29087b39-1965-453c-9d19-61a4df9b0156\") " pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j"
Nov 04 03:37:22.832422 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:22.831017    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29087b39-1965-453c-9d19-61a4df9b0156-workload-socket\") pod \"e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j\" (UID: \"29087b39-1965-453c-9d19-61a4df9b0156\") " pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j"
Nov 04 03:37:22.832422 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:22.831160    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45m98\" (UniqueName: \"kubernetes.io/projected/29087b39-1965-453c-9d19-61a4df9b0156-kube-api-access-45m98\") pod \"e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j\" (UID: \"29087b39-1965-453c-9d19-61a4df9b0156\") " pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j"
Nov 04 03:37:22.832422 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:22.831256    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29087b39-1965-453c-9d19-61a4df9b0156-istio-envoy\") pod \"e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j\" (UID: \"29087b39-1965-453c-9d19-61a4df9b0156\") " pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j"
Nov 04 03:37:22.832422 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:22.831295    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29087b39-1965-453c-9d19-61a4df9b0156-workload-certs\") pod \"e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j\" (UID: \"29087b39-1965-453c-9d19-61a4df9b0156\") " pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j"
Nov 04 03:37:22.832422 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:22.831581    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29087b39-1965-453c-9d19-61a4df9b0156-workload-socket\") pod \"e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j\" (UID: \"29087b39-1965-453c-9d19-61a4df9b0156\") " pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j"
Nov 04 03:37:22.833671 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:22.833641    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29087b39-1965-453c-9d19-61a4df9b0156-istio-podinfo\") pod \"e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j\" (UID: \"29087b39-1965-453c-9d19-61a4df9b0156\") " pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j"
Nov 04 03:37:22.837652 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:22.837628    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29087b39-1965-453c-9d19-61a4df9b0156-istio-envoy\") pod \"e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j\" (UID: \"29087b39-1965-453c-9d19-61a4df9b0156\") " pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j"
Nov 04 03:37:22.897233 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:22.896345    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-45m98\" (UniqueName: \"kubernetes.io/projected/29087b39-1965-453c-9d19-61a4df9b0156-kube-api-access-45m98\") pod \"e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j\" (UID: \"29087b39-1965-453c-9d19-61a4df9b0156\") " pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j"
Nov 04 03:37:22.945082 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:22.945047    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29087b39-1965-453c-9d19-61a4df9b0156-istio-token\") pod \"e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j\" (UID: \"29087b39-1965-453c-9d19-61a4df9b0156\") " pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j"
Nov 04 03:37:22.975816 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:22.975779    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j"
Nov 04 03:37:23.429510 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:23.429477    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j"]
Nov 04 03:37:23.438948 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:37:23.438913    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29087b39_1965_453c_9d19_61a4df9b0156.slice/crio-c944108eff2324d03659ecb1a240bfd0ea6ddd398b4e1ed6ab3bc445738d6765 WatchSource:0}: Error finding container c944108eff2324d03659ecb1a240bfd0ea6ddd398b4e1ed6ab3bc445738d6765: Status 404 returned error can't find the container with id c944108eff2324d03659ecb1a240bfd0ea6ddd398b4e1ed6ab3bc445738d6765
Nov 04 03:37:23.441946 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:23.441897    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:37:23.442071 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:23.441982    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:37:23.442071 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:23.442025    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:37:23.483587 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:23.483542    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j" event={"ID":"29087b39-1965-453c-9d19-61a4df9b0156","Type":"ContainerStarted","Data":"c944108eff2324d03659ecb1a240bfd0ea6ddd398b4e1ed6ab3bc445738d6765"}
Nov 04 03:37:24.491388 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:24.491351    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j" event={"ID":"29087b39-1965-453c-9d19-61a4df9b0156","Type":"ContainerStarted","Data":"5b5e94fd5bd8f91a89507c85487eae802f5a03c0034a9904018094c342b31060"}
Nov 04 03:37:24.656997 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:24.656941    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j" podStartSLOduration=2.656926704 podStartE2EDuration="2.656926704s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:37:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:37:24.655038256 +0000 UTC m=+1187.231691169" watchObservedRunningTime="2026-11-04 03:37:24.656926704 +0000 UTC m=+1187.233579692"
Nov 04 03:37:24.976450 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:24.976373    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j"
Nov 04 03:37:24.982671 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:24.982649    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j"
Nov 04 03:37:25.499909 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:25.499875    5609 generic.go:356] "Generic (PLEG): container finished" podID="20f015bc-d781-4317-99c3-ce697c740441" containerID="a41c54b503aa3d6a91a1fbaf959aa2a7f1d4aaba983e4222391160c984d4677d" exitCode=0
Nov 04 03:37:25.500311 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:25.499951    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v" event={"ID":"20f015bc-d781-4317-99c3-ce697c740441","Type":"ContainerDied","Data":"a41c54b503aa3d6a91a1fbaf959aa2a7f1d4aaba983e4222391160c984d4677d"}
Nov 04 03:37:25.500311 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:25.500293    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j"
Nov 04 03:37:25.500612 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:25.500597    5609 scope.go:122] "RemoveContainer" containerID="a41c54b503aa3d6a91a1fbaf959aa2a7f1d4aaba983e4222391160c984d4677d"
Nov 04 03:37:25.501385 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:25.501275    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j"
Nov 04 03:37:25.501866 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:25.501839    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:37:25.501913 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:25.501903    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:37:25.501952 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:25.501934    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:37:26.539425 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:26.538595    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v" event={"ID":"20f015bc-d781-4317-99c3-ce697c740441","Type":"ContainerStarted","Data":"afab82e4b59537fe6181b47d2e50b98096f67715d10b0eeb13141eece72da517"}
Nov 04 03:37:26.816510 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:26.811792    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v"
Nov 04 03:37:26.816510 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:26.811921    5609 patch_prober.go:29] interesting pod/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 04 03:37:26.816510 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:26.811963    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v" podUID="20f015bc-d781-4317-99c3-ce697c740441" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 04 03:37:27.501034 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:27.500996    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9"]
Nov 04 03:37:27.514502 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:27.514476    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9"
Nov 04 03:37:27.583468 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:27.570498    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-a-6d170d-openshift-default-dockercfg-4c666\""
Nov 04 03:37:27.682515 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:27.648483    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8d54648-2395-478b-940e-fca3a4deb1b1-istio-token\") pod \"e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9\" (UID: \"c8d54648-2395-478b-940e-fca3a4deb1b1\") " pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9"
Nov 04 03:37:27.682515 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:27.660557    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6zvl\" (UniqueName: \"kubernetes.io/projected/c8d54648-2395-478b-940e-fca3a4deb1b1-kube-api-access-z6zvl\") pod \"e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9\" (UID: \"c8d54648-2395-478b-940e-fca3a4deb1b1\") " pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9"
Nov 04 03:37:27.682515 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:27.670519    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8d54648-2395-478b-940e-fca3a4deb1b1-workload-certs\") pod \"e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9\" (UID: \"c8d54648-2395-478b-940e-fca3a4deb1b1\") " pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9"
Nov 04 03:37:27.682515 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:27.670628    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8d54648-2395-478b-940e-fca3a4deb1b1-istio-data\") pod \"e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9\" (UID: \"c8d54648-2395-478b-940e-fca3a4deb1b1\") " pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9"
Nov 04 03:37:27.682515 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:27.670661    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8d54648-2395-478b-940e-fca3a4deb1b1-istio-podinfo\") pod \"e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9\" (UID: \"c8d54648-2395-478b-940e-fca3a4deb1b1\") " pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9"
Nov 04 03:37:27.682515 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:27.670725    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8d54648-2395-478b-940e-fca3a4deb1b1-workload-socket\") pod \"e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9\" (UID: \"c8d54648-2395-478b-940e-fca3a4deb1b1\") " pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9"
Nov 04 03:37:27.682515 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:27.670752    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8d54648-2395-478b-940e-fca3a4deb1b1-istiod-ca-cert\") pod \"e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9\" (UID: \"c8d54648-2395-478b-940e-fca3a4deb1b1\") " pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9"
Nov 04 03:37:27.682515 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:27.670810    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8d54648-2395-478b-940e-fca3a4deb1b1-credential-socket\") pod \"e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9\" (UID: \"c8d54648-2395-478b-940e-fca3a4deb1b1\") " pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9"
Nov 04 03:37:27.682515 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:27.670880    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8d54648-2395-478b-940e-fca3a4deb1b1-istio-envoy\") pod \"e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9\" (UID: \"c8d54648-2395-478b-940e-fca3a4deb1b1\") " pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9"
Nov 04 03:37:27.756561 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:27.756478    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9"]
Nov 04 03:37:27.815800 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:27.815752    5609 patch_prober.go:29] interesting pod/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 04 03:37:27.816000 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:27.815815    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v" podUID="20f015bc-d781-4317-99c3-ce697c740441" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 04 03:37:27.818138 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:27.818103    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8d54648-2395-478b-940e-fca3a4deb1b1-istio-token\") pod \"e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9\" (UID: \"c8d54648-2395-478b-940e-fca3a4deb1b1\") " pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9"
Nov 04 03:37:27.818269 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:27.818161    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6zvl\" (UniqueName: \"kubernetes.io/projected/c8d54648-2395-478b-940e-fca3a4deb1b1-kube-api-access-z6zvl\") pod \"e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9\" (UID: \"c8d54648-2395-478b-940e-fca3a4deb1b1\") " pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9"
Nov 04 03:37:27.818345 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:27.818265    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8d54648-2395-478b-940e-fca3a4deb1b1-workload-certs\") pod \"e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9\" (UID: \"c8d54648-2395-478b-940e-fca3a4deb1b1\") " pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9"
Nov 04 03:37:27.818345 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:27.818324    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8d54648-2395-478b-940e-fca3a4deb1b1-istio-data\") pod \"e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9\" (UID: \"c8d54648-2395-478b-940e-fca3a4deb1b1\") " pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9"
Nov 04 03:37:27.818490 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:27.818349    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8d54648-2395-478b-940e-fca3a4deb1b1-istio-podinfo\") pod \"e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9\" (UID: \"c8d54648-2395-478b-940e-fca3a4deb1b1\") " pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9"
Nov 04 03:37:27.818490 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:27.818441    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8d54648-2395-478b-940e-fca3a4deb1b1-workload-socket\") pod \"e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9\" (UID: \"c8d54648-2395-478b-940e-fca3a4deb1b1\") " pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9"
Nov 04 03:37:27.818490 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:27.818474    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8d54648-2395-478b-940e-fca3a4deb1b1-istiod-ca-cert\") pod \"e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9\" (UID: \"c8d54648-2395-478b-940e-fca3a4deb1b1\") " pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9"
Nov 04 03:37:27.818658 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:27.818527    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8d54648-2395-478b-940e-fca3a4deb1b1-credential-socket\") pod \"e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9\" (UID: \"c8d54648-2395-478b-940e-fca3a4deb1b1\") " pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9"
Nov 04 03:37:27.819472 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:27.819448    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8d54648-2395-478b-940e-fca3a4deb1b1-istio-envoy\") pod \"e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9\" (UID: \"c8d54648-2395-478b-940e-fca3a4deb1b1\") " pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9"
Nov 04 03:37:27.820840 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:27.820807    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8d54648-2395-478b-940e-fca3a4deb1b1-workload-certs\") pod \"e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9\" (UID: \"c8d54648-2395-478b-940e-fca3a4deb1b1\") " pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9"
Nov 04 03:37:27.825820 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:27.825798    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8d54648-2395-478b-940e-fca3a4deb1b1-istio-podinfo\") pod \"e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9\" (UID: \"c8d54648-2395-478b-940e-fca3a4deb1b1\") " pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9"
Nov 04 03:37:27.826192 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:27.826175    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8d54648-2395-478b-940e-fca3a4deb1b1-istio-data\") pod \"e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9\" (UID: \"c8d54648-2395-478b-940e-fca3a4deb1b1\") " pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9"
Nov 04 03:37:27.826559 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:27.826525    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8d54648-2395-478b-940e-fca3a4deb1b1-credential-socket\") pod \"e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9\" (UID: \"c8d54648-2395-478b-940e-fca3a4deb1b1\") " pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9"
Nov 04 03:37:27.826653 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:27.826596    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8d54648-2395-478b-940e-fca3a4deb1b1-workload-socket\") pod \"e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9\" (UID: \"c8d54648-2395-478b-940e-fca3a4deb1b1\") " pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9"
Nov 04 03:37:27.827691 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:27.827675    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8d54648-2395-478b-940e-fca3a4deb1b1-istiod-ca-cert\") pod \"e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9\" (UID: \"c8d54648-2395-478b-940e-fca3a4deb1b1\") " pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9"
Nov 04 03:37:27.828875 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:27.828852    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8d54648-2395-478b-940e-fca3a4deb1b1-istio-envoy\") pod \"e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9\" (UID: \"c8d54648-2395-478b-940e-fca3a4deb1b1\") " pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9"
Nov 04 03:37:27.876693 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:27.876661    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8d54648-2395-478b-940e-fca3a4deb1b1-istio-token\") pod \"e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9\" (UID: \"c8d54648-2395-478b-940e-fca3a4deb1b1\") " pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9"
Nov 04 03:37:27.878370 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:27.878344    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6zvl\" (UniqueName: \"kubernetes.io/projected/c8d54648-2395-478b-940e-fca3a4deb1b1-kube-api-access-z6zvl\") pod \"e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9\" (UID: \"c8d54648-2395-478b-940e-fca3a4deb1b1\") " pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9"
Nov 04 03:37:28.128016 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:28.127929    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9"
Nov 04 03:37:28.557177 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:28.557137    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9"]
Nov 04 03:37:28.561922 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:37:28.561890    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8d54648_2395_478b_940e_fca3a4deb1b1.slice/crio-8b0b0822955d50986abbd1c7e643032d4d3477fa130ca2351c9b14f3dab36c60 WatchSource:0}: Error finding container 8b0b0822955d50986abbd1c7e643032d4d3477fa130ca2351c9b14f3dab36c60: Status 404 returned error can't find the container with id 8b0b0822955d50986abbd1c7e643032d4d3477fa130ca2351c9b14f3dab36c60
Nov 04 03:37:28.564358 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:28.564323    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:37:28.564471 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:28.564390    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:37:28.564471 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:28.564437    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:37:28.767256 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:28.767223    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9" event={"ID":"c8d54648-2395-478b-940e-fca3a4deb1b1","Type":"ContainerStarted","Data":"e7b077dbbf2a6d13cee704ad6f4079b50916b3e8cde68b168113efb40b6750f8"}
Nov 04 03:37:28.767822 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:28.767799    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9" event={"ID":"c8d54648-2395-478b-940e-fca3a4deb1b1","Type":"ContainerStarted","Data":"8b0b0822955d50986abbd1c7e643032d4d3477fa130ca2351c9b14f3dab36c60"}
Nov 04 03:37:28.807472 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:28.806844    5609 patch_prober.go:29] interesting pod/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 04 03:37:28.807472 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:28.806917    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v" podUID="20f015bc-d781-4317-99c3-ce697c740441" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 04 03:37:29.128781 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:29.128695    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9"
Nov 04 03:37:29.134512 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:29.134485    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9"
Nov 04 03:37:29.259827 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:29.259771    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9" podStartSLOduration=2.25975794 podStartE2EDuration="2.25975794s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:37:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:37:28.895752534 +0000 UTC m=+1191.472405486" watchObservedRunningTime="2026-11-04 03:37:29.25975794 +0000 UTC m=+1191.836410853"
Nov 04 03:37:29.775766 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:29.775735    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9"
Nov 04 03:37:29.776604 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:29.776584    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9"
Nov 04 03:37:29.811432 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:29.807519    5609 patch_prober.go:29] interesting pod/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 04 03:37:29.811432 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:29.807579    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v" podUID="20f015bc-d781-4317-99c3-ce697c740441" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 04 03:37:30.798880 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:30.798839    5609 patch_prober.go:29] interesting pod/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 04 03:37:30.799506 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:30.799472    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v" podUID="20f015bc-d781-4317-99c3-ce697c740441" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 04 03:37:31.803717 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:31.803686    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v"
Nov 04 03:37:31.804213 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:31.804166    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v"
Nov 04 03:37:31.805132 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:31.805109    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v"
Nov 04 03:37:33.049614 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:33.049575    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-1bf4bf-5bcc9466c5-b67d2"]
Nov 04 03:37:33.053779 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:33.053754    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-1bf4bf-5bcc9466c5-b67d2"
Nov 04 03:37:33.090912 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:33.090868    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kvlg\" (UniqueName: \"kubernetes.io/projected/b25bdf07-7285-440d-8cb5-1a34e9dea8cb-kube-api-access-4kvlg\") pod \"payload-pre-processing-e2e-extmod-b-1bf4bf-5bcc9466c5-b67d2\" (UID: \"b25bdf07-7285-440d-8cb5-1a34e9dea8cb\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-1bf4bf-5bcc9466c5-b67d2"
Nov 04 03:37:33.091109 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:33.090933    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b25bdf07-7285-440d-8cb5-1a34e9dea8cb-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-1bf4bf-5bcc9466c5-b67d2\" (UID: \"b25bdf07-7285-440d-8cb5-1a34e9dea8cb\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-1bf4bf-5bcc9466c5-b67d2"
Nov 04 03:37:33.182428 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:33.181697    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-b-1bf4bf\""
Nov 04 03:37:33.182428 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:33.181865    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-b-1bf4bf-dockercfg-m72rs\""
Nov 04 03:37:33.219452 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:33.185270    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-1bf4bf-5bcc9466c5-b67d2"]
Nov 04 03:37:33.225252 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:33.225218    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b25bdf07-7285-440d-8cb5-1a34e9dea8cb-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-1bf4bf-5bcc9466c5-b67d2\" (UID: \"b25bdf07-7285-440d-8cb5-1a34e9dea8cb\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-1bf4bf-5bcc9466c5-b67d2"
Nov 04 03:37:33.226826 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:33.226797    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kvlg\" (UniqueName: \"kubernetes.io/projected/b25bdf07-7285-440d-8cb5-1a34e9dea8cb-kube-api-access-4kvlg\") pod \"payload-pre-processing-e2e-extmod-b-1bf4bf-5bcc9466c5-b67d2\" (UID: \"b25bdf07-7285-440d-8cb5-1a34e9dea8cb\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-1bf4bf-5bcc9466c5-b67d2"
Nov 04 03:37:33.227134 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:33.226441    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b25bdf07-7285-440d-8cb5-1a34e9dea8cb-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-1bf4bf-5bcc9466c5-b67d2\" (UID: \"b25bdf07-7285-440d-8cb5-1a34e9dea8cb\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-1bf4bf-5bcc9466c5-b67d2"
Nov 04 03:37:33.288140 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:33.288110    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kvlg\" (UniqueName: \"kubernetes.io/projected/b25bdf07-7285-440d-8cb5-1a34e9dea8cb-kube-api-access-4kvlg\") pod \"payload-pre-processing-e2e-extmod-b-1bf4bf-5bcc9466c5-b67d2\" (UID: \"b25bdf07-7285-440d-8cb5-1a34e9dea8cb\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-1bf4bf-5bcc9466c5-b67d2"
Nov 04 03:37:33.369149 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:33.369060    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-1bf4bf-5bcc9466c5-b67d2"
Nov 04 03:37:33.760449 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:33.758586    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-1bf4bf-5bcc9466c5-b67d2"]
Nov 04 03:37:33.771950 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:37:33.771851    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb25bdf07_7285_440d_8cb5_1a34e9dea8cb.slice/crio-19d720d22b2bb76b68995f835a5532c5abbf16d145e217cf2b6d90e5eff80d73 WatchSource:0}: Error finding container 19d720d22b2bb76b68995f835a5532c5abbf16d145e217cf2b6d90e5eff80d73: Status 404 returned error can't find the container with id 19d720d22b2bb76b68995f835a5532c5abbf16d145e217cf2b6d90e5eff80d73
Nov 04 03:37:33.813628 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:33.813588    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-1bf4bf-5bcc9466c5-b67d2" event={"ID":"b25bdf07-7285-440d-8cb5-1a34e9dea8cb","Type":"ContainerStarted","Data":"19d720d22b2bb76b68995f835a5532c5abbf16d145e217cf2b6d90e5eff80d73"}
Nov 04 03:37:34.148269 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:34.148187    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c"]
Nov 04 03:37:34.152830 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:34.152797    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c"
Nov 04 03:37:34.233426 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:34.227704    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c"]
Nov 04 03:37:34.274427 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:34.253069    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e6b2f82-51f7-49e4-b952-7a0861e35889-workload-certs\") pod \"e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c\" (UID: \"5e6b2f82-51f7-49e4-b952-7a0861e35889\") " pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c"
Nov 04 03:37:34.274427 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:34.253146    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e6b2f82-51f7-49e4-b952-7a0861e35889-istiod-ca-cert\") pod \"e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c\" (UID: \"5e6b2f82-51f7-49e4-b952-7a0861e35889\") " pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c"
Nov 04 03:37:34.274427 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:34.253200    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e6b2f82-51f7-49e4-b952-7a0861e35889-istio-podinfo\") pod \"e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c\" (UID: \"5e6b2f82-51f7-49e4-b952-7a0861e35889\") " pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c"
Nov 04 03:37:34.274427 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:34.253251    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e6b2f82-51f7-49e4-b952-7a0861e35889-istio-envoy\") pod \"e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c\" (UID: \"5e6b2f82-51f7-49e4-b952-7a0861e35889\") " pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c"
Nov 04 03:37:34.274427 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:34.253293    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e6b2f82-51f7-49e4-b952-7a0861e35889-credential-socket\") pod \"e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c\" (UID: \"5e6b2f82-51f7-49e4-b952-7a0861e35889\") " pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c"
Nov 04 03:37:34.274427 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:34.253343    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkqk2\" (UniqueName: \"kubernetes.io/projected/5e6b2f82-51f7-49e4-b952-7a0861e35889-kube-api-access-mkqk2\") pod \"e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c\" (UID: \"5e6b2f82-51f7-49e4-b952-7a0861e35889\") " pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c"
Nov 04 03:37:34.274427 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:34.253386    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e6b2f82-51f7-49e4-b952-7a0861e35889-istio-data\") pod \"e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c\" (UID: \"5e6b2f82-51f7-49e4-b952-7a0861e35889\") " pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c"
Nov 04 03:37:34.274427 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:34.253448    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e6b2f82-51f7-49e4-b952-7a0861e35889-istio-token\") pod \"e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c\" (UID: \"5e6b2f82-51f7-49e4-b952-7a0861e35889\") " pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c"
Nov 04 03:37:34.274427 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:34.253486    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e6b2f82-51f7-49e4-b952-7a0861e35889-workload-socket\") pod \"e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c\" (UID: \"5e6b2f82-51f7-49e4-b952-7a0861e35889\") " pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c"
Nov 04 03:37:34.302426 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:34.301796    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-b-59c311-openshift-default-dockercfg-pvsn5\""
Nov 04 03:37:34.359352 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:34.356552    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e6b2f82-51f7-49e4-b952-7a0861e35889-workload-certs\") pod \"e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c\" (UID: \"5e6b2f82-51f7-49e4-b952-7a0861e35889\") " pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c"
Nov 04 03:37:34.359352 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:34.356611    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e6b2f82-51f7-49e4-b952-7a0861e35889-istiod-ca-cert\") pod \"e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c\" (UID: \"5e6b2f82-51f7-49e4-b952-7a0861e35889\") " pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c"
Nov 04 03:37:34.359352 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:34.356644    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e6b2f82-51f7-49e4-b952-7a0861e35889-istio-podinfo\") pod \"e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c\" (UID: \"5e6b2f82-51f7-49e4-b952-7a0861e35889\") " pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c"
Nov 04 03:37:34.359352 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:34.356677    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e6b2f82-51f7-49e4-b952-7a0861e35889-istio-envoy\") pod \"e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c\" (UID: \"5e6b2f82-51f7-49e4-b952-7a0861e35889\") " pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c"
Nov 04 03:37:34.359352 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:34.356705    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e6b2f82-51f7-49e4-b952-7a0861e35889-credential-socket\") pod \"e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c\" (UID: \"5e6b2f82-51f7-49e4-b952-7a0861e35889\") " pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c"
Nov 04 03:37:34.359352 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:34.356738    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkqk2\" (UniqueName: \"kubernetes.io/projected/5e6b2f82-51f7-49e4-b952-7a0861e35889-kube-api-access-mkqk2\") pod \"e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c\" (UID: \"5e6b2f82-51f7-49e4-b952-7a0861e35889\") " pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c"
Nov 04 03:37:34.359352 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:34.356768    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e6b2f82-51f7-49e4-b952-7a0861e35889-istio-data\") pod \"e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c\" (UID: \"5e6b2f82-51f7-49e4-b952-7a0861e35889\") " pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c"
Nov 04 03:37:34.359352 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:34.356803    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e6b2f82-51f7-49e4-b952-7a0861e35889-istio-token\") pod \"e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c\" (UID: \"5e6b2f82-51f7-49e4-b952-7a0861e35889\") " pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c"
Nov 04 03:37:34.359352 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:34.356840    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e6b2f82-51f7-49e4-b952-7a0861e35889-workload-socket\") pod \"e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c\" (UID: \"5e6b2f82-51f7-49e4-b952-7a0861e35889\") " pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c"
Nov 04 03:37:34.359352 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:34.357344    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e6b2f82-51f7-49e4-b952-7a0861e35889-workload-socket\") pod \"e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c\" (UID: \"5e6b2f82-51f7-49e4-b952-7a0861e35889\") " pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c"
Nov 04 03:37:34.359352 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:34.357611    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e6b2f82-51f7-49e4-b952-7a0861e35889-workload-certs\") pod \"e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c\" (UID: \"5e6b2f82-51f7-49e4-b952-7a0861e35889\") " pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c"
Nov 04 03:37:34.359352 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:34.358237    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e6b2f82-51f7-49e4-b952-7a0861e35889-istiod-ca-cert\") pod \"e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c\" (UID: \"5e6b2f82-51f7-49e4-b952-7a0861e35889\") " pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c"
Nov 04 03:37:34.362863 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:34.362553    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e6b2f82-51f7-49e4-b952-7a0861e35889-istio-data\") pod \"e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c\" (UID: \"5e6b2f82-51f7-49e4-b952-7a0861e35889\") " pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c"
Nov 04 03:37:34.363275 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:34.363240    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e6b2f82-51f7-49e4-b952-7a0861e35889-credential-socket\") pod \"e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c\" (UID: \"5e6b2f82-51f7-49e4-b952-7a0861e35889\") " pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c"
Nov 04 03:37:34.368832 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:34.368741    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e6b2f82-51f7-49e4-b952-7a0861e35889-istio-podinfo\") pod \"e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c\" (UID: \"5e6b2f82-51f7-49e4-b952-7a0861e35889\") " pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c"
Nov 04 03:37:34.383003 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:34.382709    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e6b2f82-51f7-49e4-b952-7a0861e35889-istio-envoy\") pod \"e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c\" (UID: \"5e6b2f82-51f7-49e4-b952-7a0861e35889\") " pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c"
Nov 04 03:37:34.427812 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:34.427768    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e6b2f82-51f7-49e4-b952-7a0861e35889-istio-token\") pod \"e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c\" (UID: \"5e6b2f82-51f7-49e4-b952-7a0861e35889\") " pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c"
Nov 04 03:37:34.429545 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:34.429481    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkqk2\" (UniqueName: \"kubernetes.io/projected/5e6b2f82-51f7-49e4-b952-7a0861e35889-kube-api-access-mkqk2\") pod \"e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c\" (UID: \"5e6b2f82-51f7-49e4-b952-7a0861e35889\") " pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c"
Nov 04 03:37:34.470517 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:34.470482    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c"
Nov 04 03:37:34.849472 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:34.849144    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-1bf4bf-5bcc9466c5-b67d2" event={"ID":"b25bdf07-7285-440d-8cb5-1a34e9dea8cb","Type":"ContainerStarted","Data":"8607dd7ed0bc104d40135f56eeca8842c050178a3f76c2d07ca99050ffb66395"}
Nov 04 03:37:34.850277 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:34.850201    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-1bf4bf-5bcc9466c5-b67d2"
Nov 04 03:37:34.963728 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:34.963672    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-1bf4bf-5bcc9466c5-b67d2" podStartSLOduration=1.9636566370000001 podStartE2EDuration="1.963656637s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:37:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:37:34.960492318 +0000 UTC m=+1197.537145233" watchObservedRunningTime="2026-11-04 03:37:34.963656637 +0000 UTC m=+1197.540309552"
Nov 04 03:37:35.001979 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:35.001945    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c"]
Nov 04 03:37:35.042698 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:35.042652    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:37:35.042952 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:35.042923    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:37:35.048513 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:35.048479    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:37:35.859813 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:35.859774    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c" event={"ID":"5e6b2f82-51f7-49e4-b952-7a0861e35889","Type":"ContainerStarted","Data":"64a5416ece825642bebb2ea435c8bfc743e4c4e9133eeab52d306482b9545b1a"}
Nov 04 03:37:35.860234 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:35.859838    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c" event={"ID":"5e6b2f82-51f7-49e4-b952-7a0861e35889","Type":"ContainerStarted","Data":"10aa41ddc5533ea407bad7817f4a40b280db638880068d258f294b70f410c0f3"}
Nov 04 03:37:35.976833 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:35.976776    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c" podStartSLOduration=1.976760875 podStartE2EDuration="1.976760875s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:37:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:37:35.975090385 +0000 UTC m=+1198.551743298" watchObservedRunningTime="2026-11-04 03:37:35.976760875 +0000 UTC m=+1198.553413829"
Nov 04 03:37:36.471337 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:36.471305    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c"
Nov 04 03:37:36.476825 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:36.476800    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c"
Nov 04 03:37:36.867576 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:36.867498    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c"
Nov 04 03:37:36.868427 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:36.868377    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c"
Nov 04 03:37:37.790936 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:37.790892    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-p9989_068619f8-7fbb-4b22-ac7d-a1f144e084c0/console-operator/2.log"
Nov 04 03:37:37.798745 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:37.798723    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vmw5z_21a42913-5c4e-4057-aae5-2804038f16e4/ovn-acl-logging/0.log"
Nov 04 03:37:37.800047 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:37.800026    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-p9989_068619f8-7fbb-4b22-ac7d-a1f144e084c0/console-operator/2.log"
Nov 04 03:37:37.807557 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:37.807534    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vmw5z_21a42913-5c4e-4057-aae5-2804038f16e4/ovn-acl-logging/0.log"
Nov 04 03:37:37.971317 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:37.971285    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-6d170d-5847588f6c-sp8v2"]
Nov 04 03:37:37.975854 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:37.975832    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6d170d-5847588f6c-sp8v2"
Nov 04 03:37:38.027434 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:38.026695    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-a-6d170d-dockercfg-8fssf\""
Nov 04 03:37:38.027434 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:38.026779    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-a-6d170d\""
Nov 04 03:37:38.027434 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:38.026918    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52ckd\" (UniqueName: \"kubernetes.io/projected/4da4148c-c27f-4b6a-af35-543b1f81fb15-kube-api-access-52ckd\") pod \"payload-pre-processing-e2e-extmod-a-6d170d-5847588f6c-sp8v2\" (UID: \"4da4148c-c27f-4b6a-af35-543b1f81fb15\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6d170d-5847588f6c-sp8v2"
Nov 04 03:37:38.027434 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:38.027002    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4da4148c-c27f-4b6a-af35-543b1f81fb15-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-6d170d-5847588f6c-sp8v2\" (UID: \"4da4148c-c27f-4b6a-af35-543b1f81fb15\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6d170d-5847588f6c-sp8v2"
Nov 04 03:37:38.031946 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:38.031920    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-6d170d-5847588f6c-sp8v2"]
Nov 04 03:37:38.155422 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:38.154762    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52ckd\" (UniqueName: \"kubernetes.io/projected/4da4148c-c27f-4b6a-af35-543b1f81fb15-kube-api-access-52ckd\") pod \"payload-pre-processing-e2e-extmod-a-6d170d-5847588f6c-sp8v2\" (UID: \"4da4148c-c27f-4b6a-af35-543b1f81fb15\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6d170d-5847588f6c-sp8v2"
Nov 04 03:37:38.155604 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:38.155472    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4da4148c-c27f-4b6a-af35-543b1f81fb15-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-6d170d-5847588f6c-sp8v2\" (UID: \"4da4148c-c27f-4b6a-af35-543b1f81fb15\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6d170d-5847588f6c-sp8v2"
Nov 04 03:37:38.167422 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:38.159096    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4da4148c-c27f-4b6a-af35-543b1f81fb15-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-6d170d-5847588f6c-sp8v2\" (UID: \"4da4148c-c27f-4b6a-af35-543b1f81fb15\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6d170d-5847588f6c-sp8v2"
Nov 04 03:37:38.216121 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:38.216086    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-52ckd\" (UniqueName: \"kubernetes.io/projected/4da4148c-c27f-4b6a-af35-543b1f81fb15-kube-api-access-52ckd\") pod \"payload-pre-processing-e2e-extmod-a-6d170d-5847588f6c-sp8v2\" (UID: \"4da4148c-c27f-4b6a-af35-543b1f81fb15\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6d170d-5847588f6c-sp8v2"
Nov 04 03:37:38.290310 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:38.290226    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6d170d-5847588f6c-sp8v2"
Nov 04 03:37:38.666448 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:38.666347    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-6d170d-5847588f6c-sp8v2"]
Nov 04 03:37:38.670766 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:37:38.670742    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4da4148c_c27f_4b6a_af35_543b1f81fb15.slice/crio-51e2f43d0a9d01752cb299e2453e1963dd99ee3326349a08a31cba5b61550fc1 WatchSource:0}: Error finding container 51e2f43d0a9d01752cb299e2453e1963dd99ee3326349a08a31cba5b61550fc1: Status 404 returned error can't find the container with id 51e2f43d0a9d01752cb299e2453e1963dd99ee3326349a08a31cba5b61550fc1
Nov 04 03:37:38.926453 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:38.926331    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6d170d-5847588f6c-sp8v2" event={"ID":"4da4148c-c27f-4b6a-af35-543b1f81fb15","Type":"ContainerStarted","Data":"2abe406ac007c72bf9f702b1cf0a42734630f1509d2514943c0c587e3d5eb1e7"}
Nov 04 03:37:38.926453 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:38.926437    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6d170d-5847588f6c-sp8v2" event={"ID":"4da4148c-c27f-4b6a-af35-543b1f81fb15","Type":"ContainerStarted","Data":"51e2f43d0a9d01752cb299e2453e1963dd99ee3326349a08a31cba5b61550fc1"}
Nov 04 03:37:38.926668 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:38.926653    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6d170d-5847588f6c-sp8v2"
Nov 04 03:37:39.040028 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:39.039978    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6d170d-5847588f6c-sp8v2" podStartSLOduration=2.039963313 podStartE2EDuration="2.039963313s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:37:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:37:39.036411852 +0000 UTC m=+1201.613064760" watchObservedRunningTime="2026-11-04 03:37:39.039963313 +0000 UTC m=+1201.616616227"
Nov 04 03:37:45.863731 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:45.863703    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-1bf4bf-5bcc9466c5-b67d2"
Nov 04 03:37:48.560437 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:37:48.560374    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:37:49.936142 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:49.936113    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6d170d-5847588f6c-sp8v2"
Nov 04 03:37:52.427358 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:52.427325    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-59c311-77bc87bfc9-xvvgl"]
Nov 04 03:37:52.431721 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:52.431702    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-59c311-77bc87bfc9-xvvgl"
Nov 04 03:37:52.463292 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:52.463258    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmtv7\" (UniqueName: \"kubernetes.io/projected/43ac1896-22eb-4e13-889f-250eeb780886-kube-api-access-mmtv7\") pod \"payload-pre-processing-e2e-extmod-b-59c311-77bc87bfc9-xvvgl\" (UID: \"43ac1896-22eb-4e13-889f-250eeb780886\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-59c311-77bc87bfc9-xvvgl"
Nov 04 03:37:52.463478 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:52.463457    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43ac1896-22eb-4e13-889f-250eeb780886-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-59c311-77bc87bfc9-xvvgl\" (UID: \"43ac1896-22eb-4e13-889f-250eeb780886\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-59c311-77bc87bfc9-xvvgl"
Nov 04 03:37:52.485068 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:52.485037    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-59c311-77bc87bfc9-xvvgl"]
Nov 04 03:37:52.534677 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:52.534641    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-b-59c311\""
Nov 04 03:37:52.570484 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:52.568807    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-b-59c311-dockercfg-h6rml\""
Nov 04 03:37:52.571065 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:52.571024    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43ac1896-22eb-4e13-889f-250eeb780886-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-59c311-77bc87bfc9-xvvgl\" (UID: \"43ac1896-22eb-4e13-889f-250eeb780886\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-59c311-77bc87bfc9-xvvgl"
Nov 04 03:37:52.571208 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:52.571186    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmtv7\" (UniqueName: \"kubernetes.io/projected/43ac1896-22eb-4e13-889f-250eeb780886-kube-api-access-mmtv7\") pod \"payload-pre-processing-e2e-extmod-b-59c311-77bc87bfc9-xvvgl\" (UID: \"43ac1896-22eb-4e13-889f-250eeb780886\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-59c311-77bc87bfc9-xvvgl"
Nov 04 03:37:52.579204 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:52.572598    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43ac1896-22eb-4e13-889f-250eeb780886-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-59c311-77bc87bfc9-xvvgl\" (UID: \"43ac1896-22eb-4e13-889f-250eeb780886\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-59c311-77bc87bfc9-xvvgl"
Nov 04 03:37:52.639844 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:52.639809    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmtv7\" (UniqueName: \"kubernetes.io/projected/43ac1896-22eb-4e13-889f-250eeb780886-kube-api-access-mmtv7\") pod \"payload-pre-processing-e2e-extmod-b-59c311-77bc87bfc9-xvvgl\" (UID: \"43ac1896-22eb-4e13-889f-250eeb780886\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-59c311-77bc87bfc9-xvvgl"
Nov 04 03:37:52.745688 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:52.745604    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-59c311-77bc87bfc9-xvvgl"
Nov 04 03:37:53.151085 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:53.151003    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-59c311-77bc87bfc9-xvvgl"]
Nov 04 03:37:53.155569 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:37:53.155534    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43ac1896_22eb_4e13_889f_250eeb780886.slice/crio-16086e2b1230855a930a4726fe3304fc725c90628edc0eb56f7f82b882803342 WatchSource:0}: Error finding container 16086e2b1230855a930a4726fe3304fc725c90628edc0eb56f7f82b882803342: Status 404 returned error can't find the container with id 16086e2b1230855a930a4726fe3304fc725c90628edc0eb56f7f82b882803342
Nov 04 03:37:54.045581 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:54.045544    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-59c311-77bc87bfc9-xvvgl" event={"ID":"43ac1896-22eb-4e13-889f-250eeb780886","Type":"ContainerStarted","Data":"a77198f873f1ffc36f522945291e11383e9cdf58da0aa3f934fcced12f7f9eab"}
Nov 04 03:37:54.045989 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:54.045601    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-59c311-77bc87bfc9-xvvgl" event={"ID":"43ac1896-22eb-4e13-889f-250eeb780886","Type":"ContainerStarted","Data":"16086e2b1230855a930a4726fe3304fc725c90628edc0eb56f7f82b882803342"}
Nov 04 03:37:54.045989 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:54.045630    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-59c311-77bc87bfc9-xvvgl"
Nov 04 03:37:54.157990 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:54.157938    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-59c311-77bc87bfc9-xvvgl" podStartSLOduration=2.157923401 podStartE2EDuration="2.157923401s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:37:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:37:54.155260454 +0000 UTC m=+1216.731913369" watchObservedRunningTime="2026-11-04 03:37:54.157923401 +0000 UTC m=+1216.734576365"
Nov 04 03:37:59.216765 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.216725    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw"]
Nov 04 03:37:59.220085 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.220063    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw"
Nov 04 03:37:59.263686 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.263655    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s"]
Nov 04 03:37:59.266811 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.266791    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s"
Nov 04 03:37:59.369063 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.369025    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dd4f492-1db4-4368-b0b3-23661c65b6d2-workload-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw\" (UID: \"7dd4f492-1db4-4368-b0b3-23661c65b6d2\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw"
Nov 04 03:37:59.369254 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.369090    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-credential-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s\" (UID: \"1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s"
Nov 04 03:37:59.369254 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.369207    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7dd4f492-1db4-4368-b0b3-23661c65b6d2-workload-certs\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw\" (UID: \"7dd4f492-1db4-4368-b0b3-23661c65b6d2\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw"
Nov 04 03:37:59.369254 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.369242    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7dd4f492-1db4-4368-b0b3-23661c65b6d2-istio-token\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw\" (UID: \"7dd4f492-1db4-4368-b0b3-23661c65b6d2\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw"
Nov 04 03:37:59.369389 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.369363    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-istiod-ca-cert\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s\" (UID: \"1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s"
Nov 04 03:37:59.369453 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.369436    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-workload-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s\" (UID: \"1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s"
Nov 04 03:37:59.369511 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.369490    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-istio-envoy\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s\" (UID: \"1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s"
Nov 04 03:37:59.369563 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.369533    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k95wm\" (UniqueName: \"kubernetes.io/projected/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-kube-api-access-k95wm\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s\" (UID: \"1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s"
Nov 04 03:37:59.369626 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.369609    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7dd4f492-1db4-4368-b0b3-23661c65b6d2-istio-envoy\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw\" (UID: \"7dd4f492-1db4-4368-b0b3-23661c65b6d2\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw"
Nov 04 03:37:59.369659 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.369643    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7dd4f492-1db4-4368-b0b3-23661c65b6d2-istiod-ca-cert\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw\" (UID: \"7dd4f492-1db4-4368-b0b3-23661c65b6d2\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw"
Nov 04 03:37:59.369694 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.369672    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-workload-certs\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s\" (UID: \"1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s"
Nov 04 03:37:59.369726 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.369705    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7dd4f492-1db4-4368-b0b3-23661c65b6d2-istio-data\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw\" (UID: \"7dd4f492-1db4-4368-b0b3-23661c65b6d2\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw"
Nov 04 03:37:59.369759 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.369739    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-istio-podinfo\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s\" (UID: \"1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s"
Nov 04 03:37:59.369810 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.369781    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7dd4f492-1db4-4368-b0b3-23661c65b6d2-istio-podinfo\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw\" (UID: \"7dd4f492-1db4-4368-b0b3-23661c65b6d2\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw"
Nov 04 03:37:59.369873 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.369850    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dd4f492-1db4-4368-b0b3-23661c65b6d2-credential-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw\" (UID: \"7dd4f492-1db4-4368-b0b3-23661c65b6d2\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw"
Nov 04 03:37:59.369926 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.369879    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbg67\" (UniqueName: \"kubernetes.io/projected/7dd4f492-1db4-4368-b0b3-23661c65b6d2-kube-api-access-qbg67\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw\" (UID: \"7dd4f492-1db4-4368-b0b3-23661c65b6d2\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw"
Nov 04 03:37:59.369967 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.369924    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-istio-data\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s\" (UID: \"1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s"
Nov 04 03:37:59.370001 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.369964    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-istio-token\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s\" (UID: \"1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s"
Nov 04 03:37:59.421365 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.421326    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw"]
Nov 04 03:37:59.471466 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.471362    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7dd4f492-1db4-4368-b0b3-23661c65b6d2-workload-certs\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw\" (UID: \"7dd4f492-1db4-4368-b0b3-23661c65b6d2\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw"
Nov 04 03:37:59.471466 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.471412    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7dd4f492-1db4-4368-b0b3-23661c65b6d2-istio-token\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw\" (UID: \"7dd4f492-1db4-4368-b0b3-23661c65b6d2\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw"
Nov 04 03:37:59.471643 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.471499    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-istiod-ca-cert\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s\" (UID: \"1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s"
Nov 04 03:37:59.471643 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.471549    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-workload-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s\" (UID: \"1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s"
Nov 04 03:37:59.471643 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.471577    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-istio-envoy\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s\" (UID: \"1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s"
Nov 04 03:37:59.471643 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.471613    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k95wm\" (UniqueName: \"kubernetes.io/projected/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-kube-api-access-k95wm\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s\" (UID: \"1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s"
Nov 04 03:37:59.471836 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.471656    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7dd4f492-1db4-4368-b0b3-23661c65b6d2-istio-envoy\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw\" (UID: \"7dd4f492-1db4-4368-b0b3-23661c65b6d2\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw"
Nov 04 03:37:59.471836 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.471678    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7dd4f492-1db4-4368-b0b3-23661c65b6d2-istiod-ca-cert\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw\" (UID: \"7dd4f492-1db4-4368-b0b3-23661c65b6d2\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw"
Nov 04 03:37:59.471836 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.471709    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-workload-certs\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s\" (UID: \"1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s"
Nov 04 03:37:59.471836 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.471737    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7dd4f492-1db4-4368-b0b3-23661c65b6d2-istio-data\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw\" (UID: \"7dd4f492-1db4-4368-b0b3-23661c65b6d2\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw"
Nov 04 03:37:59.471836 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.471772    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-istio-podinfo\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s\" (UID: \"1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s"
Nov 04 03:37:59.471836 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.471802    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7dd4f492-1db4-4368-b0b3-23661c65b6d2-istio-podinfo\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw\" (UID: \"7dd4f492-1db4-4368-b0b3-23661c65b6d2\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw"
Nov 04 03:37:59.471836 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.471808    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7dd4f492-1db4-4368-b0b3-23661c65b6d2-workload-certs\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw\" (UID: \"7dd4f492-1db4-4368-b0b3-23661c65b6d2\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw"
Nov 04 03:37:59.471836 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.471832    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dd4f492-1db4-4368-b0b3-23661c65b6d2-credential-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw\" (UID: \"7dd4f492-1db4-4368-b0b3-23661c65b6d2\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw"
Nov 04 03:37:59.472230 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.471859    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbg67\" (UniqueName: \"kubernetes.io/projected/7dd4f492-1db4-4368-b0b3-23661c65b6d2-kube-api-access-qbg67\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw\" (UID: \"7dd4f492-1db4-4368-b0b3-23661c65b6d2\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw"
Nov 04 03:37:59.472230 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.471913    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-istio-data\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s\" (UID: \"1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s"
Nov 04 03:37:59.472230 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.471945    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-istio-token\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s\" (UID: \"1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s"
Nov 04 03:37:59.472230 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.471997    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dd4f492-1db4-4368-b0b3-23661c65b6d2-workload-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw\" (UID: \"7dd4f492-1db4-4368-b0b3-23661c65b6d2\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw"
Nov 04 03:37:59.472230 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.472021    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-workload-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s\" (UID: \"1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s"
Nov 04 03:37:59.472230 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.472098    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7dd4f492-1db4-4368-b0b3-23661c65b6d2-istio-data\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw\" (UID: \"7dd4f492-1db4-4368-b0b3-23661c65b6d2\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw"
Nov 04 03:37:59.472230 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.472110    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-credential-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s\" (UID: \"1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s"
Nov 04 03:37:59.472609 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.472228    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-workload-certs\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s\" (UID: \"1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s"
Nov 04 03:37:59.472609 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.472435    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-credential-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s\" (UID: \"1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s"
Nov 04 03:37:59.472609 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.472435    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7dd4f492-1db4-4368-b0b3-23661c65b6d2-istiod-ca-cert\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw\" (UID: \"7dd4f492-1db4-4368-b0b3-23661c65b6d2\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw"
Nov 04 03:37:59.472609 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.472567    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dd4f492-1db4-4368-b0b3-23661c65b6d2-credential-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw\" (UID: \"7dd4f492-1db4-4368-b0b3-23661c65b6d2\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw"
Nov 04 03:37:59.472923 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.472728    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dd4f492-1db4-4368-b0b3-23661c65b6d2-workload-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw\" (UID: \"7dd4f492-1db4-4368-b0b3-23661c65b6d2\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw"
Nov 04 03:37:59.472923 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.472846    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-istio-data\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s\" (UID: \"1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s"
Nov 04 03:37:59.473160 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.473136    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-istiod-ca-cert\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s\" (UID: \"1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s"
Nov 04 03:37:59.474233 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.474208    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7dd4f492-1db4-4368-b0b3-23661c65b6d2-istio-envoy\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw\" (UID: \"7dd4f492-1db4-4368-b0b3-23661c65b6d2\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw"
Nov 04 03:37:59.474233 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.474218    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-istio-envoy\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s\" (UID: \"1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s"
Nov 04 03:37:59.474691 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.474671    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7dd4f492-1db4-4368-b0b3-23661c65b6d2-istio-podinfo\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw\" (UID: \"7dd4f492-1db4-4368-b0b3-23661c65b6d2\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw"
Nov 04 03:37:59.474780 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.474734    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-istio-podinfo\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s\" (UID: \"1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s"
Nov 04 03:37:59.530881 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.530852    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-istio-token\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s\" (UID: \"1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s"
Nov 04 03:37:59.531033 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.531010    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbg67\" (UniqueName: \"kubernetes.io/projected/7dd4f492-1db4-4368-b0b3-23661c65b6d2-kube-api-access-qbg67\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw\" (UID: \"7dd4f492-1db4-4368-b0b3-23661c65b6d2\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw"
Nov 04 03:37:59.531077 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.531024    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k95wm\" (UniqueName: \"kubernetes.io/projected/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-kube-api-access-k95wm\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s\" (UID: \"1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s"
Nov 04 03:37:59.531632 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.531612    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7dd4f492-1db4-4368-b0b3-23661c65b6d2-istio-token\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw\" (UID: \"7dd4f492-1db4-4368-b0b3-23661c65b6d2\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw"
Nov 04 03:37:59.533667 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.533646    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw"
Nov 04 03:37:59.549252 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.549227    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s"]
Nov 04 03:37:59.580815 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.580772    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s"
Nov 04 03:37:59.972659 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.972627    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw"]
Nov 04 03:37:59.976438 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:37:59.976412    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dd4f492_1db4_4368_b0b3_23661c65b6d2.slice/crio-49411c703a4e2c63b2fb2c644f4044d39ea2ced742b35425aa211892b3d078a2 WatchSource:0}: Error finding container 49411c703a4e2c63b2fb2c644f4044d39ea2ced742b35425aa211892b3d078a2: Status 404 returned error can't find the container with id 49411c703a4e2c63b2fb2c644f4044d39ea2ced742b35425aa211892b3d078a2
Nov 04 03:37:59.978671 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.978642    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:37:59.978730 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.978707    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:37:59.978762 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:37:59.978737    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:38:00.027118 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:00.027093    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s"]
Nov 04 03:38:00.034742 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:38:00.034716    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fa79428_b3e5_4d0f_a8f8_6f6d40e84e99.slice/crio-b1e72012fa50d6db2486a13c65b5025f51dc30ecef8de136c7066769d2d55bd5 WatchSource:0}: Error finding container b1e72012fa50d6db2486a13c65b5025f51dc30ecef8de136c7066769d2d55bd5: Status 404 returned error can't find the container with id b1e72012fa50d6db2486a13c65b5025f51dc30ecef8de136c7066769d2d55bd5
Nov 04 03:38:00.037781 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:00.037740    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:38:00.038024 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:00.037826    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:38:00.038024 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:00.037863    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:38:00.092452 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:00.092395    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s" event={"ID":"1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99","Type":"ContainerStarted","Data":"b1e72012fa50d6db2486a13c65b5025f51dc30ecef8de136c7066769d2d55bd5"}
Nov 04 03:38:00.094199 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:00.094169    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw" event={"ID":"7dd4f492-1db4-4368-b0b3-23661c65b6d2","Type":"ContainerStarted","Data":"7e46a85065f2720af67542ad6f53d5bdbe6713b35b67e9301882040a3f277aed"}
Nov 04 03:38:00.094334 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:00.094214    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw" event={"ID":"7dd4f492-1db4-4368-b0b3-23661c65b6d2","Type":"ContainerStarted","Data":"49411c703a4e2c63b2fb2c644f4044d39ea2ced742b35425aa211892b3d078a2"}
Nov 04 03:38:00.217262 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:00.217210    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw" podStartSLOduration=1.217192976 podStartE2EDuration="1.217192976s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:37:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:38:00.211950226 +0000 UTC m=+1222.788603140" watchObservedRunningTime="2026-11-04 03:38:00.217192976 +0000 UTC m=+1222.793845892"
Nov 04 03:38:00.533985 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:00.533951    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw"
Nov 04 03:38:01.133841 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:01.133807    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s" event={"ID":"1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99","Type":"ContainerStarted","Data":"9af6ef4cce350d3c87edfd993ebc5fe4a97c95e904e48f986790da22a526a49f"}
Nov 04 03:38:01.317414 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:01.317333    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s" podStartSLOduration=2.317314118 podStartE2EDuration="2.317314118s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:37:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:38:01.316970156 +0000 UTC m=+1223.893623072" watchObservedRunningTime="2026-11-04 03:38:01.317314118 +0000 UTC m=+1223.893967033"
Nov 04 03:38:01.539690 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:01.539648    5609 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Nov 04 03:38:01.539889 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:01.539725    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw" podUID="7dd4f492-1db4-4368-b0b3-23661c65b6d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Nov 04 03:38:01.581350 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:01.581322    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s"
Nov 04 03:38:01.582789 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:01.582759    5609 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Nov 04 03:38:01.582905 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:01.582810    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s" podUID="1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Nov 04 03:38:02.536910 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:02.536871    5609 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Nov 04 03:38:02.537530 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:02.537483    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw" podUID="7dd4f492-1db4-4368-b0b3-23661c65b6d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Nov 04 03:38:02.581378 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:02.581293    5609 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Nov 04 03:38:02.581378 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:02.581353    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s" podUID="1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Nov 04 03:38:03.534055 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:03.534018    5609 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Nov 04 03:38:03.534334 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:03.534306    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw" podUID="7dd4f492-1db4-4368-b0b3-23661c65b6d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Nov 04 03:38:03.581462 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:03.581429    5609 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Nov 04 03:38:03.581820 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:03.581478    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s" podUID="1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Nov 04 03:38:04.537782 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:04.537741    5609 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Nov 04 03:38:04.538074 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:04.538043    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw" podUID="7dd4f492-1db4-4368-b0b3-23661c65b6d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Nov 04 03:38:04.581257 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:04.581232    5609 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Nov 04 03:38:04.581461 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:04.581441    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s" podUID="1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Nov 04 03:38:05.055347 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:05.055320    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-59c311-77bc87bfc9-xvvgl"
Nov 04 03:38:05.534838 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:05.534800    5609 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Nov 04 03:38:05.535134 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:05.535102    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw" podUID="7dd4f492-1db4-4368-b0b3-23661c65b6d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Nov 04 03:38:05.581317 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:05.581282    5609 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Nov 04 03:38:05.581482 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:05.581347    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s" podUID="1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Nov 04 03:38:06.534628 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:06.534593    5609 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Nov 04 03:38:06.535002 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:06.534643    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw" podUID="7dd4f492-1db4-4368-b0b3-23661c65b6d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Nov 04 03:38:06.581970 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:06.581939    5609 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Nov 04 03:38:06.582130 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:06.581991    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s" podUID="1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Nov 04 03:38:07.539113 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:07.539085    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw"
Nov 04 03:38:07.539495 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:07.539434    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw"
Nov 04 03:38:07.540448 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:07.540421    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw"
Nov 04 03:38:07.585073 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:07.585048    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s"
Nov 04 03:38:07.585304 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:07.585285    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s"
Nov 04 03:38:07.585937 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:07.585920    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s"
Nov 04 03:38:07.666887 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:07.666857    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-1bf4bf-689dcc8854-wv6z4"]
Nov 04 03:38:07.671176 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:07.671158    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-1bf4bf-689dcc8854-wv6z4"
Nov 04 03:38:07.727743 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:07.727714    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11207c06-566c-452c-a515-6d095242d4b5-metrics-certs\") pod \"maas-api-e2e-extmod-b-1bf4bf-689dcc8854-wv6z4\" (UID: \"11207c06-566c-452c-a515-6d095242d4b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-1bf4bf-689dcc8854-wv6z4"
Nov 04 03:38:07.727909 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:07.727754    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11207c06-566c-452c-a515-6d095242d4b5-maas-api-tls\") pod \"maas-api-e2e-extmod-b-1bf4bf-689dcc8854-wv6z4\" (UID: \"11207c06-566c-452c-a515-6d095242d4b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-1bf4bf-689dcc8854-wv6z4"
Nov 04 03:38:07.727909 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:07.727889    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11207c06-566c-452c-a515-6d095242d4b5-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-1bf4bf-689dcc8854-wv6z4\" (UID: \"11207c06-566c-452c-a515-6d095242d4b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-1bf4bf-689dcc8854-wv6z4"
Nov 04 03:38:07.728018 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:07.727947    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m66t8\" (UniqueName: \"kubernetes.io/projected/11207c06-566c-452c-a515-6d095242d4b5-kube-api-access-m66t8\") pod \"maas-api-e2e-extmod-b-1bf4bf-689dcc8854-wv6z4\" (UID: \"11207c06-566c-452c-a515-6d095242d4b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-1bf4bf-689dcc8854-wv6z4"
Nov 04 03:38:07.747754 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:07.747726    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-b-1bf4bf-serving-cert\""
Nov 04 03:38:07.753031 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:07.753007    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-1bf4bf-689dcc8854-wv6z4"]
Nov 04 03:38:07.834075 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:07.833989    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11207c06-566c-452c-a515-6d095242d4b5-metrics-certs\") pod \"maas-api-e2e-extmod-b-1bf4bf-689dcc8854-wv6z4\" (UID: \"11207c06-566c-452c-a515-6d095242d4b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-1bf4bf-689dcc8854-wv6z4"
Nov 04 03:38:07.834301 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:07.834279    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11207c06-566c-452c-a515-6d095242d4b5-maas-api-tls\") pod \"maas-api-e2e-extmod-b-1bf4bf-689dcc8854-wv6z4\" (UID: \"11207c06-566c-452c-a515-6d095242d4b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-1bf4bf-689dcc8854-wv6z4"
Nov 04 03:38:07.834499 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:07.834481    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11207c06-566c-452c-a515-6d095242d4b5-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-1bf4bf-689dcc8854-wv6z4\" (UID: \"11207c06-566c-452c-a515-6d095242d4b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-1bf4bf-689dcc8854-wv6z4"
Nov 04 03:38:07.834618 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:07.834604    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m66t8\" (UniqueName: \"kubernetes.io/projected/11207c06-566c-452c-a515-6d095242d4b5-kube-api-access-m66t8\") pod \"maas-api-e2e-extmod-b-1bf4bf-689dcc8854-wv6z4\" (UID: \"11207c06-566c-452c-a515-6d095242d4b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-1bf4bf-689dcc8854-wv6z4"
Nov 04 03:38:07.835536 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:38:07.835513    5609 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-b-1bf4bf-serving-cert: secret "maas-api-e2e-extmod-b-1bf4bf-serving-cert" not found
Nov 04 03:38:07.836647 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:38:07.836631    5609 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11207c06-566c-452c-a515-6d095242d4b5-maas-api-tls podName:11207c06-566c-452c-a515-6d095242d4b5 nodeName:}" failed. No retries permitted until 2026-11-04 03:38:08.336609206 +0000 UTC m=+1230.913262099 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/11207c06-566c-452c-a515-6d095242d4b5-maas-api-tls") pod "maas-api-e2e-extmod-b-1bf4bf-689dcc8854-wv6z4" (UID: "11207c06-566c-452c-a515-6d095242d4b5") : secret "maas-api-e2e-extmod-b-1bf4bf-serving-cert" not found
Nov 04 03:38:07.846621 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:07.836526    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11207c06-566c-452c-a515-6d095242d4b5-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-1bf4bf-689dcc8854-wv6z4\" (UID: \"11207c06-566c-452c-a515-6d095242d4b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-1bf4bf-689dcc8854-wv6z4"
Nov 04 03:38:07.847478 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:07.847336    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11207c06-566c-452c-a515-6d095242d4b5-metrics-certs\") pod \"maas-api-e2e-extmod-b-1bf4bf-689dcc8854-wv6z4\" (UID: \"11207c06-566c-452c-a515-6d095242d4b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-1bf4bf-689dcc8854-wv6z4"
Nov 04 03:38:07.905792 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:07.905763    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m66t8\" (UniqueName: \"kubernetes.io/projected/11207c06-566c-452c-a515-6d095242d4b5-kube-api-access-m66t8\") pod \"maas-api-e2e-extmod-b-1bf4bf-689dcc8854-wv6z4\" (UID: \"11207c06-566c-452c-a515-6d095242d4b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-1bf4bf-689dcc8854-wv6z4"
Nov 04 03:38:08.341937 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:08.341898    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11207c06-566c-452c-a515-6d095242d4b5-maas-api-tls\") pod \"maas-api-e2e-extmod-b-1bf4bf-689dcc8854-wv6z4\" (UID: \"11207c06-566c-452c-a515-6d095242d4b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-1bf4bf-689dcc8854-wv6z4"
Nov 04 03:38:08.344353 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:08.344333    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11207c06-566c-452c-a515-6d095242d4b5-maas-api-tls\") pod \"maas-api-e2e-extmod-b-1bf4bf-689dcc8854-wv6z4\" (UID: \"11207c06-566c-452c-a515-6d095242d4b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-1bf4bf-689dcc8854-wv6z4"
Nov 04 03:38:08.585987 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:08.585954    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-1bf4bf-689dcc8854-wv6z4"
Nov 04 03:38:08.961292 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:08.961257    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-1bf4bf-689dcc8854-wv6z4"]
Nov 04 03:38:08.965968 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:38:08.965943    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11207c06_566c_452c_a515_6d095242d4b5.slice/crio-64985c82bb14c39b28c63f17cb8faeaef3c2d6f052e34a4ffeadb927240d99de WatchSource:0}: Error finding container 64985c82bb14c39b28c63f17cb8faeaef3c2d6f052e34a4ffeadb927240d99de: Status 404 returned error can't find the container with id 64985c82bb14c39b28c63f17cb8faeaef3c2d6f052e34a4ffeadb927240d99de
Nov 04 03:38:09.197954 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:09.197917    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-1bf4bf-689dcc8854-wv6z4" event={"ID":"11207c06-566c-452c-a515-6d095242d4b5","Type":"ContainerStarted","Data":"64985c82bb14c39b28c63f17cb8faeaef3c2d6f052e34a4ffeadb927240d99de"}
Nov 04 03:38:10.219838 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:10.219801    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-1bf4bf-689dcc8854-wv6z4" event={"ID":"11207c06-566c-452c-a515-6d095242d4b5","Type":"ContainerStarted","Data":"b9313d1b7e27d519e5128c35615eb615b7cec6e01f5d2fffe4f1cdeceff28023"}
Nov 04 03:38:10.220235 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:10.220007    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-1bf4bf-689dcc8854-wv6z4"
Nov 04 03:38:10.334421 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:10.334295    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-1bf4bf-689dcc8854-wv6z4" podStartSLOduration=2.273055392 podStartE2EDuration="3.334280952s" totalImagesPullingTime="1.06122556s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:38:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:38:10.332852676 +0000 UTC m=+1232.909505590" watchObservedRunningTime="2026-11-04 03:38:10.334280952 +0000 UTC m=+1232.910933865"
Nov 04 03:38:16.232531 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:16.232502    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-1bf4bf-689dcc8854-wv6z4"
Nov 04 03:38:18.560027 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:38:18.559986    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:38:32.338425 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:32.336500    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b"]
Nov 04 03:38:32.349905 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:32.349876    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b"
Nov 04 03:38:32.365340 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:32.364990    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e2b497b-98dc-4fa4-bbb5-a357f795628d-istio-envoy\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b\" (UID: \"3e2b497b-98dc-4fa4-bbb5-a357f795628d\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b"
Nov 04 03:38:32.365340 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:32.365069    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e2b497b-98dc-4fa4-bbb5-a357f795628d-istio-token\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b\" (UID: \"3e2b497b-98dc-4fa4-bbb5-a357f795628d\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b"
Nov 04 03:38:32.365340 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:32.365206    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e2b497b-98dc-4fa4-bbb5-a357f795628d-istio-data\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b\" (UID: \"3e2b497b-98dc-4fa4-bbb5-a357f795628d\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b"
Nov 04 03:38:32.365340 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:32.365322    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e2b497b-98dc-4fa4-bbb5-a357f795628d-workload-certs\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b\" (UID: \"3e2b497b-98dc-4fa4-bbb5-a357f795628d\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b"
Nov 04 03:38:32.365598 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:32.365348    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e2b497b-98dc-4fa4-bbb5-a357f795628d-workload-socket\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b\" (UID: \"3e2b497b-98dc-4fa4-bbb5-a357f795628d\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b"
Nov 04 03:38:32.365598 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:32.365444    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e2b497b-98dc-4fa4-bbb5-a357f795628d-istio-podinfo\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b\" (UID: \"3e2b497b-98dc-4fa4-bbb5-a357f795628d\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b"
Nov 04 03:38:32.365598 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:32.365477    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e2b497b-98dc-4fa4-bbb5-a357f795628d-istiod-ca-cert\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b\" (UID: \"3e2b497b-98dc-4fa4-bbb5-a357f795628d\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b"
Nov 04 03:38:32.365598 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:32.365500    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e2b497b-98dc-4fa4-bbb5-a357f795628d-credential-socket\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b\" (UID: \"3e2b497b-98dc-4fa4-bbb5-a357f795628d\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b"
Nov 04 03:38:32.365598 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:32.365543    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sj7t\" (UniqueName: \"kubernetes.io/projected/3e2b497b-98dc-4fa4-bbb5-a357f795628d-kube-api-access-9sj7t\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b\" (UID: \"3e2b497b-98dc-4fa4-bbb5-a357f795628d\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b"
Nov 04 03:38:32.466386 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:32.466343    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e2b497b-98dc-4fa4-bbb5-a357f795628d-istio-envoy\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b\" (UID: \"3e2b497b-98dc-4fa4-bbb5-a357f795628d\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b"
Nov 04 03:38:32.466727 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:32.466688    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e2b497b-98dc-4fa4-bbb5-a357f795628d-istio-token\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b\" (UID: \"3e2b497b-98dc-4fa4-bbb5-a357f795628d\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b"
Nov 04 03:38:32.466876 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:32.466431    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b"]
Nov 04 03:38:32.467024 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:32.467007    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e2b497b-98dc-4fa4-bbb5-a357f795628d-istio-data\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b\" (UID: \"3e2b497b-98dc-4fa4-bbb5-a357f795628d\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b"
Nov 04 03:38:32.467235 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:32.467215    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e2b497b-98dc-4fa4-bbb5-a357f795628d-workload-certs\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b\" (UID: \"3e2b497b-98dc-4fa4-bbb5-a357f795628d\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b"
Nov 04 03:38:32.467495 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:32.467377    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e2b497b-98dc-4fa4-bbb5-a357f795628d-workload-socket\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b\" (UID: \"3e2b497b-98dc-4fa4-bbb5-a357f795628d\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b"
Nov 04 03:38:32.467694 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:32.467659    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e2b497b-98dc-4fa4-bbb5-a357f795628d-istio-podinfo\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b\" (UID: \"3e2b497b-98dc-4fa4-bbb5-a357f795628d\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b"
Nov 04 03:38:32.467858 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:32.467837    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e2b497b-98dc-4fa4-bbb5-a357f795628d-workload-certs\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b\" (UID: \"3e2b497b-98dc-4fa4-bbb5-a357f795628d\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b"
Nov 04 03:38:32.467944 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:32.467843    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e2b497b-98dc-4fa4-bbb5-a357f795628d-istiod-ca-cert\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b\" (UID: \"3e2b497b-98dc-4fa4-bbb5-a357f795628d\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b"
Nov 04 03:38:32.467944 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:32.467912    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e2b497b-98dc-4fa4-bbb5-a357f795628d-credential-socket\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b\" (UID: \"3e2b497b-98dc-4fa4-bbb5-a357f795628d\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b"
Nov 04 03:38:32.468049 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:32.467977    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9sj7t\" (UniqueName: \"kubernetes.io/projected/3e2b497b-98dc-4fa4-bbb5-a357f795628d-kube-api-access-9sj7t\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b\" (UID: \"3e2b497b-98dc-4fa4-bbb5-a357f795628d\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b"
Nov 04 03:38:32.468447 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:32.467571    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e2b497b-98dc-4fa4-bbb5-a357f795628d-istio-data\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b\" (UID: \"3e2b497b-98dc-4fa4-bbb5-a357f795628d\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b"
Nov 04 03:38:32.468697 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:32.468677    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e2b497b-98dc-4fa4-bbb5-a357f795628d-workload-socket\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b\" (UID: \"3e2b497b-98dc-4fa4-bbb5-a357f795628d\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b"
Nov 04 03:38:32.469172 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:32.469149    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e2b497b-98dc-4fa4-bbb5-a357f795628d-credential-socket\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b\" (UID: \"3e2b497b-98dc-4fa4-bbb5-a357f795628d\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b"
Nov 04 03:38:32.469270 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:32.469224    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e2b497b-98dc-4fa4-bbb5-a357f795628d-istiod-ca-cert\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b\" (UID: \"3e2b497b-98dc-4fa4-bbb5-a357f795628d\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b"
Nov 04 03:38:32.474054 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:32.473911    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e2b497b-98dc-4fa4-bbb5-a357f795628d-istio-podinfo\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b\" (UID: \"3e2b497b-98dc-4fa4-bbb5-a357f795628d\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b"
Nov 04 03:38:32.481313 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:32.481291    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e2b497b-98dc-4fa4-bbb5-a357f795628d-istio-envoy\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b\" (UID: \"3e2b497b-98dc-4fa4-bbb5-a357f795628d\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b"
Nov 04 03:38:32.561348 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:32.561278    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e2b497b-98dc-4fa4-bbb5-a357f795628d-istio-token\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b\" (UID: \"3e2b497b-98dc-4fa4-bbb5-a357f795628d\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b"
Nov 04 03:38:32.563025 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:32.562985    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sj7t\" (UniqueName: \"kubernetes.io/projected/3e2b497b-98dc-4fa4-bbb5-a357f795628d-kube-api-access-9sj7t\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b\" (UID: \"3e2b497b-98dc-4fa4-bbb5-a357f795628d\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b"
Nov 04 03:38:32.674073 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:32.673992    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b"
Nov 04 03:38:33.479225 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:33.478673    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:38:33.479225 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:33.478763    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:38:33.479225 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:33.478811    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:38:33.479225 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:33.479029    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b"]
Nov 04 03:38:34.516437 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:34.515984    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b" event={"ID":"3e2b497b-98dc-4fa4-bbb5-a357f795628d","Type":"ContainerStarted","Data":"52113e6d5858fda7d97ed01e115b705fc8dcad081c482d8bfdcb0129e97fd285"}
Nov 04 03:38:34.516437 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:34.516073    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b" event={"ID":"3e2b497b-98dc-4fa4-bbb5-a357f795628d","Type":"ContainerStarted","Data":"80b8c6ef0d8da20f8907c08a92552b7f1ed0c9402befded34071a1c4e4267f33"}
Nov 04 03:38:34.674762 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:34.674729    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b"
Nov 04 03:38:34.675869 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:34.675843    5609 patch_prober.go:29] interesting pod/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Nov 04 03:38:34.675942 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:34.675895    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b" podUID="3e2b497b-98dc-4fa4-bbb5-a357f795628d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Nov 04 03:38:34.740426 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:34.732078    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b" podStartSLOduration=2.732056057 podStartE2EDuration="2.732056057s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:38:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:38:34.699012985 +0000 UTC m=+1257.275665898" watchObservedRunningTime="2026-11-04 03:38:34.732056057 +0000 UTC m=+1257.308708972"
Nov 04 03:38:35.678446 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:35.676493    5609 patch_prober.go:29] interesting pod/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Nov 04 03:38:35.678446 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:35.676551    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b" podUID="3e2b497b-98dc-4fa4-bbb5-a357f795628d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Nov 04 03:38:36.674744 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:36.674710    5609 patch_prober.go:29] interesting pod/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Nov 04 03:38:36.674887 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:36.674776    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b" podUID="3e2b497b-98dc-4fa4-bbb5-a357f795628d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Nov 04 03:38:37.675286 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:37.675248    5609 patch_prober.go:29] interesting pod/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Nov 04 03:38:37.675667 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:37.675307    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b" podUID="3e2b497b-98dc-4fa4-bbb5-a357f795628d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Nov 04 03:38:38.675130 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:38.675090    5609 patch_prober.go:29] interesting pod/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Nov 04 03:38:38.675312 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:38.675151    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b" podUID="3e2b497b-98dc-4fa4-bbb5-a357f795628d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Nov 04 03:38:39.675561 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:39.675513    5609 patch_prober.go:29] interesting pod/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Nov 04 03:38:39.676075 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:39.675590    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b" podUID="3e2b497b-98dc-4fa4-bbb5-a357f795628d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Nov 04 03:38:40.675231 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:40.675187    5609 patch_prober.go:29] interesting pod/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Nov 04 03:38:40.675442 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:40.675261    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b" podUID="3e2b497b-98dc-4fa4-bbb5-a357f795628d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Nov 04 03:38:41.546571 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:41.546522    5609 patch_prober.go:29] interesting pod/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 03:38:41.547090 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:41.546599    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j" podUID="29087b39-1965-453c-9d19-61a4df9b0156" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 03:38:41.675235 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:41.675197    5609 patch_prober.go:29] interesting pod/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Nov 04 03:38:42.355767 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:41.675246    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b" podUID="3e2b497b-98dc-4fa4-bbb5-a357f795628d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Nov 04 03:38:42.680352 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:42.680327    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b"
Nov 04 03:38:42.680839 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:42.680818    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b"
Nov 04 03:38:42.681513 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:42.681494    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b"
Nov 04 03:38:42.720249 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:42.720221    5609 generic.go:356] "Generic (PLEG): container finished" podID="c8d54648-2395-478b-940e-fca3a4deb1b1" containerID="e7b077dbbf2a6d13cee704ad6f4079b50916b3e8cde68b168113efb40b6750f8" exitCode=0
Nov 04 03:38:42.720380 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:42.720293    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9" event={"ID":"c8d54648-2395-478b-940e-fca3a4deb1b1","Type":"ContainerDied","Data":"e7b077dbbf2a6d13cee704ad6f4079b50916b3e8cde68b168113efb40b6750f8"}
Nov 04 03:38:42.720702 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:42.720686    5609 scope.go:122] "RemoveContainer" containerID="e7b077dbbf2a6d13cee704ad6f4079b50916b3e8cde68b168113efb40b6750f8"
Nov 04 03:38:42.722096 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:42.722068    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:38:42.722172 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:42.722104    5609 generic.go:356] "Generic (PLEG): container finished" podID="29087b39-1965-453c-9d19-61a4df9b0156" containerID="5b5e94fd5bd8f91a89507c85487eae802f5a03c0034a9904018094c342b31060" exitCode=0
Nov 04 03:38:42.722172 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:42.722137    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:38:42.722242 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:42.722177    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:38:42.722242 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:42.722186    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j" event={"ID":"29087b39-1965-453c-9d19-61a4df9b0156","Type":"ContainerDied","Data":"5b5e94fd5bd8f91a89507c85487eae802f5a03c0034a9904018094c342b31060"}
Nov 04 03:38:42.722734 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:42.722626    5609 scope.go:122] "RemoveContainer" containerID="5b5e94fd5bd8f91a89507c85487eae802f5a03c0034a9904018094c342b31060"
Nov 04 03:38:42.723832 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:42.723804    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:38:42.723929 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:42.723863    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:38:42.723929 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:42.723893    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:38:43.747333 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:43.747294    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j" event={"ID":"29087b39-1965-453c-9d19-61a4df9b0156","Type":"ContainerStarted","Data":"40e265ab2cc5138d454dbdecdf48920dea22d886f8f0056f51b5fc9acf1b0e63"}
Nov 04 03:38:43.749515 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:43.749491    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9" event={"ID":"c8d54648-2395-478b-940e-fca3a4deb1b1","Type":"ContainerStarted","Data":"fdb46f7c87147cecb74f89c8179759932224f4ca5e15b25c80b91781c74912ad"}
Nov 04 03:38:43.976647 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:43.976617    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j"
Nov 04 03:38:43.976914 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:43.976895    5609 patch_prober.go:29] interesting pod/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 04 03:38:43.976960 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:43.976939    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j" podUID="29087b39-1965-453c-9d19-61a4df9b0156" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 04 03:38:44.134556 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:44.134468    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9"
Nov 04 03:38:44.135950 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:44.135922    5609 patch_prober.go:29] interesting pod/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 04 03:38:44.136078 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:44.135971    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9" podUID="c8d54648-2395-478b-940e-fca3a4deb1b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 04 03:38:44.775781 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:44.775730    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9"
Nov 04 03:38:44.976226 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:44.976191    5609 patch_prober.go:29] interesting pod/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 04 03:38:44.976393 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:44.976241    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j" podUID="29087b39-1965-453c-9d19-61a4df9b0156" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 04 03:38:45.135125 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:45.134895    5609 patch_prober.go:29] interesting pod/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 04 03:38:45.135125 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:45.134980    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9" podUID="c8d54648-2395-478b-940e-fca3a4deb1b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 04 03:38:45.977042 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:45.977005    5609 patch_prober.go:29] interesting pod/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 04 03:38:45.977430 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:45.977072    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j" podUID="29087b39-1965-453c-9d19-61a4df9b0156" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 04 03:38:46.131807 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:46.131768    5609 patch_prober.go:29] interesting pod/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 04 03:38:46.132092 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:46.132058    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9" podUID="c8d54648-2395-478b-940e-fca3a4deb1b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 04 03:38:46.976536 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:46.976500    5609 patch_prober.go:29] interesting pod/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 04 03:38:46.976711 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:46.976551    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j" podUID="29087b39-1965-453c-9d19-61a4df9b0156" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 04 03:38:47.133313 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:47.133280    5609 patch_prober.go:29] interesting pod/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 04 03:38:47.133921 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:47.133889    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9" podUID="c8d54648-2395-478b-940e-fca3a4deb1b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 04 03:38:47.976112 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:47.976032    5609 patch_prober.go:29] interesting pod/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 04 03:38:47.976347 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:47.976314    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j" podUID="29087b39-1965-453c-9d19-61a4df9b0156" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 04 03:38:48.136373 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:48.136335    5609 patch_prober.go:29] interesting pod/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 04 03:38:48.136980 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:48.136945    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9" podUID="c8d54648-2395-478b-940e-fca3a4deb1b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 04 03:38:48.619642 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:38:48.619603    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:38:49.041827 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:49.041793    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j"
Nov 04 03:38:49.044865 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:49.042889    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j"
Nov 04 03:38:49.044865 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:49.044081    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j"
Nov 04 03:38:49.133570 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:49.133539    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9"
Nov 04 03:38:49.134396 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:38:49.134378    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9"
Nov 04 03:39:06.868381 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:06.868343    5609 patch_prober.go:29] interesting pod/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 04 03:39:06.868853 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:06.868439    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c" podUID="5e6b2f82-51f7-49e4-b952-7a0861e35889" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 04 03:39:06.967964 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:06.967926    5609 generic.go:356] "Generic (PLEG): container finished" podID="5e6b2f82-51f7-49e4-b952-7a0861e35889" containerID="64a5416ece825642bebb2ea435c8bfc743e4c4e9133eeab52d306482b9545b1a" exitCode=0
Nov 04 03:39:06.968145 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:06.967962    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c" event={"ID":"5e6b2f82-51f7-49e4-b952-7a0861e35889","Type":"ContainerDied","Data":"64a5416ece825642bebb2ea435c8bfc743e4c4e9133eeab52d306482b9545b1a"}
Nov 04 03:39:06.968455 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:06.968438    5609 scope.go:122] "RemoveContainer" containerID="64a5416ece825642bebb2ea435c8bfc743e4c4e9133eeab52d306482b9545b1a"
Nov 04 03:39:06.969992 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:06.969961    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:39:06.970102 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:06.970045    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:39:06.970102 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:06.970095    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:39:07.264875 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.264840    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk"]
Nov 04 03:39:07.268655 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.268624    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk"
Nov 04 03:39:07.295108 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.295077    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-26xvk"]
Nov 04 03:39:07.297983 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.297957    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd1ae838-b620-411d-bd66-82c7d32cc41a-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-hgwbk\" (UID: \"fd1ae838-b620-411d-bd66-82c7d32cc41a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk"
Nov 04 03:39:07.298109 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.297998    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd1ae838-b620-411d-bd66-82c7d32cc41a-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-hgwbk\" (UID: \"fd1ae838-b620-411d-bd66-82c7d32cc41a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk"
Nov 04 03:39:07.298109 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.298021    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd1ae838-b620-411d-bd66-82c7d32cc41a-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-hgwbk\" (UID: \"fd1ae838-b620-411d-bd66-82c7d32cc41a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk"
Nov 04 03:39:07.298221 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.298176    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xsmp\" (UniqueName: \"kubernetes.io/projected/fd1ae838-b620-411d-bd66-82c7d32cc41a-kube-api-access-4xsmp\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-hgwbk\" (UID: \"fd1ae838-b620-411d-bd66-82c7d32cc41a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk"
Nov 04 03:39:07.298304 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.298287    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd1ae838-b620-411d-bd66-82c7d32cc41a-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-hgwbk\" (UID: \"fd1ae838-b620-411d-bd66-82c7d32cc41a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk"
Nov 04 03:39:07.298363 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.298351    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd1ae838-b620-411d-bd66-82c7d32cc41a-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-hgwbk\" (UID: \"fd1ae838-b620-411d-bd66-82c7d32cc41a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk"
Nov 04 03:39:07.298559 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.298507    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd1ae838-b620-411d-bd66-82c7d32cc41a-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-hgwbk\" (UID: \"fd1ae838-b620-411d-bd66-82c7d32cc41a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk"
Nov 04 03:39:07.298635 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.298617    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd1ae838-b620-411d-bd66-82c7d32cc41a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-hgwbk\" (UID: \"fd1ae838-b620-411d-bd66-82c7d32cc41a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk"
Nov 04 03:39:07.298776 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.298758    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd1ae838-b620-411d-bd66-82c7d32cc41a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-hgwbk\" (UID: \"fd1ae838-b620-411d-bd66-82c7d32cc41a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk"
Nov 04 03:39:07.298879 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.298861    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-26xvk"
Nov 04 03:39:07.327742 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.327703    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk"]
Nov 04 03:39:07.379530 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.379494    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-26xvk"]
Nov 04 03:39:07.400784 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.400744    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/621375f0-75dd-445b-8e50-50b52600a7dc-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-26xvk\" (UID: \"621375f0-75dd-445b-8e50-50b52600a7dc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-26xvk"
Nov 04 03:39:07.400974 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.400796    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/621375f0-75dd-445b-8e50-50b52600a7dc-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-26xvk\" (UID: \"621375f0-75dd-445b-8e50-50b52600a7dc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-26xvk"
Nov 04 03:39:07.400974 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.400838    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/621375f0-75dd-445b-8e50-50b52600a7dc-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-26xvk\" (UID: \"621375f0-75dd-445b-8e50-50b52600a7dc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-26xvk"
Nov 04 03:39:07.400974 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.400908    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd1ae838-b620-411d-bd66-82c7d32cc41a-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-hgwbk\" (UID: \"fd1ae838-b620-411d-bd66-82c7d32cc41a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk"
Nov 04 03:39:07.400974 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.400949    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd1ae838-b620-411d-bd66-82c7d32cc41a-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-hgwbk\" (UID: \"fd1ae838-b620-411d-bd66-82c7d32cc41a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk"
Nov 04 03:39:07.401469 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.401434    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd1ae838-b620-411d-bd66-82c7d32cc41a-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-hgwbk\" (UID: \"fd1ae838-b620-411d-bd66-82c7d32cc41a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk"
Nov 04 03:39:07.401580 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.401508    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/621375f0-75dd-445b-8e50-50b52600a7dc-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-26xvk\" (UID: \"621375f0-75dd-445b-8e50-50b52600a7dc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-26xvk"
Nov 04 03:39:07.401580 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.401536    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/621375f0-75dd-445b-8e50-50b52600a7dc-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-26xvk\" (UID: \"621375f0-75dd-445b-8e50-50b52600a7dc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-26xvk"
Nov 04 03:39:07.401580 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.401567    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xsmp\" (UniqueName: \"kubernetes.io/projected/fd1ae838-b620-411d-bd66-82c7d32cc41a-kube-api-access-4xsmp\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-hgwbk\" (UID: \"fd1ae838-b620-411d-bd66-82c7d32cc41a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk"
Nov 04 03:39:07.401747 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.401616    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/621375f0-75dd-445b-8e50-50b52600a7dc-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-26xvk\" (UID: \"621375f0-75dd-445b-8e50-50b52600a7dc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-26xvk"
Nov 04 03:39:07.401747 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.401684    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd1ae838-b620-411d-bd66-82c7d32cc41a-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-hgwbk\" (UID: \"fd1ae838-b620-411d-bd66-82c7d32cc41a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk"
Nov 04 03:39:07.401747 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.401713    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/621375f0-75dd-445b-8e50-50b52600a7dc-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-26xvk\" (UID: \"621375f0-75dd-445b-8e50-50b52600a7dc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-26xvk"
Nov 04 03:39:07.401894 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.401755    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd1ae838-b620-411d-bd66-82c7d32cc41a-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-hgwbk\" (UID: \"fd1ae838-b620-411d-bd66-82c7d32cc41a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk"
Nov 04 03:39:07.401894 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.401784    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/621375f0-75dd-445b-8e50-50b52600a7dc-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-26xvk\" (UID: \"621375f0-75dd-445b-8e50-50b52600a7dc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-26xvk"
Nov 04 03:39:07.401894 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.401819    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd1ae838-b620-411d-bd66-82c7d32cc41a-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-hgwbk\" (UID: \"fd1ae838-b620-411d-bd66-82c7d32cc41a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk"
Nov 04 03:39:07.401894 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.401844    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd1ae838-b620-411d-bd66-82c7d32cc41a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-hgwbk\" (UID: \"fd1ae838-b620-411d-bd66-82c7d32cc41a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk"
Nov 04 03:39:07.401894 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.401889    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktfh8\" (UniqueName: \"kubernetes.io/projected/621375f0-75dd-445b-8e50-50b52600a7dc-kube-api-access-ktfh8\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-26xvk\" (UID: \"621375f0-75dd-445b-8e50-50b52600a7dc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-26xvk"
Nov 04 03:39:07.402143 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.402012    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd1ae838-b620-411d-bd66-82c7d32cc41a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-hgwbk\" (UID: \"fd1ae838-b620-411d-bd66-82c7d32cc41a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk"
Nov 04 03:39:07.403604 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.403481    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd1ae838-b620-411d-bd66-82c7d32cc41a-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-hgwbk\" (UID: \"fd1ae838-b620-411d-bd66-82c7d32cc41a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk"
Nov 04 03:39:07.405342 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.403935    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd1ae838-b620-411d-bd66-82c7d32cc41a-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-hgwbk\" (UID: \"fd1ae838-b620-411d-bd66-82c7d32cc41a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk"
Nov 04 03:39:07.405342 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.404167    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd1ae838-b620-411d-bd66-82c7d32cc41a-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-hgwbk\" (UID: \"fd1ae838-b620-411d-bd66-82c7d32cc41a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk"
Nov 04 03:39:07.405342 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.405051    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd1ae838-b620-411d-bd66-82c7d32cc41a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-hgwbk\" (UID: \"fd1ae838-b620-411d-bd66-82c7d32cc41a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk"
Nov 04 03:39:07.405342 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.405298    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd1ae838-b620-411d-bd66-82c7d32cc41a-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-hgwbk\" (UID: \"fd1ae838-b620-411d-bd66-82c7d32cc41a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk"
Nov 04 03:39:07.418259 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.418207    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd1ae838-b620-411d-bd66-82c7d32cc41a-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-hgwbk\" (UID: \"fd1ae838-b620-411d-bd66-82c7d32cc41a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk"
Nov 04 03:39:07.422065 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.422034    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd1ae838-b620-411d-bd66-82c7d32cc41a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-hgwbk\" (UID: \"fd1ae838-b620-411d-bd66-82c7d32cc41a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk"
Nov 04 03:39:07.519470 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.519369    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktfh8\" (UniqueName: \"kubernetes.io/projected/621375f0-75dd-445b-8e50-50b52600a7dc-kube-api-access-ktfh8\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-26xvk\" (UID: \"621375f0-75dd-445b-8e50-50b52600a7dc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-26xvk"
Nov 04 03:39:07.519828 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.519806    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/621375f0-75dd-445b-8e50-50b52600a7dc-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-26xvk\" (UID: \"621375f0-75dd-445b-8e50-50b52600a7dc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-26xvk"
Nov 04 03:39:07.519977 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.519960    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/621375f0-75dd-445b-8e50-50b52600a7dc-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-26xvk\" (UID: \"621375f0-75dd-445b-8e50-50b52600a7dc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-26xvk"
Nov 04 03:39:07.520103 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.520091    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/621375f0-75dd-445b-8e50-50b52600a7dc-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-26xvk\" (UID: \"621375f0-75dd-445b-8e50-50b52600a7dc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-26xvk"
Nov 04 03:39:07.520293 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.520278    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/621375f0-75dd-445b-8e50-50b52600a7dc-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-26xvk\" (UID: \"621375f0-75dd-445b-8e50-50b52600a7dc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-26xvk"
Nov 04 03:39:07.520389 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.520378    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/621375f0-75dd-445b-8e50-50b52600a7dc-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-26xvk\" (UID: \"621375f0-75dd-445b-8e50-50b52600a7dc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-26xvk"
Nov 04 03:39:07.520525 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.520510    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/621375f0-75dd-445b-8e50-50b52600a7dc-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-26xvk\" (UID: \"621375f0-75dd-445b-8e50-50b52600a7dc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-26xvk"
Nov 04 03:39:07.520682 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.520656    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/621375f0-75dd-445b-8e50-50b52600a7dc-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-26xvk\" (UID: \"621375f0-75dd-445b-8e50-50b52600a7dc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-26xvk"
Nov 04 03:39:07.520799 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.520782    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/621375f0-75dd-445b-8e50-50b52600a7dc-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-26xvk\" (UID: \"621375f0-75dd-445b-8e50-50b52600a7dc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-26xvk"
Nov 04 03:39:07.521472 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.521443    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/621375f0-75dd-445b-8e50-50b52600a7dc-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-26xvk\" (UID: \"621375f0-75dd-445b-8e50-50b52600a7dc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-26xvk"
Nov 04 03:39:07.587490 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.575236    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/621375f0-75dd-445b-8e50-50b52600a7dc-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-26xvk\" (UID: \"621375f0-75dd-445b-8e50-50b52600a7dc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-26xvk"
Nov 04 03:39:07.587490 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.575633    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/621375f0-75dd-445b-8e50-50b52600a7dc-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-26xvk\" (UID: \"621375f0-75dd-445b-8e50-50b52600a7dc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-26xvk"
Nov 04 03:39:07.587490 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.577711    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/621375f0-75dd-445b-8e50-50b52600a7dc-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-26xvk\" (UID: \"621375f0-75dd-445b-8e50-50b52600a7dc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-26xvk"
Nov 04 03:39:07.587490 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.578941    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/621375f0-75dd-445b-8e50-50b52600a7dc-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-26xvk\" (UID: \"621375f0-75dd-445b-8e50-50b52600a7dc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-26xvk"
Nov 04 03:39:07.587490 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.584336    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd1ae838-b620-411d-bd66-82c7d32cc41a-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-hgwbk\" (UID: \"fd1ae838-b620-411d-bd66-82c7d32cc41a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk"
Nov 04 03:39:07.587490 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.586837    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/621375f0-75dd-445b-8e50-50b52600a7dc-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-26xvk\" (UID: \"621375f0-75dd-445b-8e50-50b52600a7dc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-26xvk"
Nov 04 03:39:07.587490 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.587189    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/621375f0-75dd-445b-8e50-50b52600a7dc-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-26xvk\" (UID: \"621375f0-75dd-445b-8e50-50b52600a7dc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-26xvk"
Nov 04 03:39:07.588200 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.588178    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xsmp\" (UniqueName: \"kubernetes.io/projected/fd1ae838-b620-411d-bd66-82c7d32cc41a-kube-api-access-4xsmp\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-hgwbk\" (UID: \"fd1ae838-b620-411d-bd66-82c7d32cc41a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk"
Nov 04 03:39:07.612380 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.612341    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktfh8\" (UniqueName: \"kubernetes.io/projected/621375f0-75dd-445b-8e50-50b52600a7dc-kube-api-access-ktfh8\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-26xvk\" (UID: \"621375f0-75dd-445b-8e50-50b52600a7dc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-26xvk"
Nov 04 03:39:07.648425 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.642898    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/621375f0-75dd-445b-8e50-50b52600a7dc-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-26xvk\" (UID: \"621375f0-75dd-445b-8e50-50b52600a7dc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-26xvk"
Nov 04 03:39:07.889747 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.889657    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk"
Nov 04 03:39:07.914101 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:07.914069    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-26xvk"
Nov 04 03:39:08.034296 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:08.034259    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c" event={"ID":"5e6b2f82-51f7-49e4-b952-7a0861e35889","Type":"ContainerStarted","Data":"43726a00aff279cf31edf98af79ed074a9ed86a859b55b9a72f6cc113ab2bb3b"}
Nov 04 03:39:08.348449 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:08.348379    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk"]
Nov 04 03:39:08.352617 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:39:08.352590    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd1ae838_b620_411d_bd66_82c7d32cc41a.slice/crio-6f274359d7cd414342db203279a4a332acab0caf6b6f2479d75dfe007525bfae WatchSource:0}: Error finding container 6f274359d7cd414342db203279a4a332acab0caf6b6f2479d75dfe007525bfae: Status 404 returned error can't find the container with id 6f274359d7cd414342db203279a4a332acab0caf6b6f2479d75dfe007525bfae
Nov 04 03:39:08.354635 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:08.354607    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-26xvk"]
Nov 04 03:39:08.354830 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:39:08.354801    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod621375f0_75dd_445b_8e50_50b52600a7dc.slice/crio-0331ce5bc759b0b1dd2612eb8108aba4a2421ab41fbf4b43bee3067ce1e7d9fc WatchSource:0}: Error finding container 0331ce5bc759b0b1dd2612eb8108aba4a2421ab41fbf4b43bee3067ce1e7d9fc: Status 404 returned error can't find the container with id 0331ce5bc759b0b1dd2612eb8108aba4a2421ab41fbf4b43bee3067ce1e7d9fc
Nov 04 03:39:08.356800 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:08.356758    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:39:08.356874 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:08.356843    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:39:08.356915 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:08.356881    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:39:08.357355 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:08.357324    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:39:08.357539 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:08.357382    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:39:08.357539 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:08.357424    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:39:08.471883 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:08.471755    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c"
Nov 04 03:39:08.471883 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:08.471795    5609 patch_prober.go:29] interesting pod/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 04 03:39:08.471883 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:08.471846    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c" podUID="5e6b2f82-51f7-49e4-b952-7a0861e35889" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 04 03:39:09.046755 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:09.046714    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk" event={"ID":"fd1ae838-b620-411d-bd66-82c7d32cc41a","Type":"ContainerStarted","Data":"26fb5458439a00c1e3812f6a6ce6c0b18c936870030b63c3795d2ac33aaea851"}
Nov 04 03:39:09.047253 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:09.046790    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk" event={"ID":"fd1ae838-b620-411d-bd66-82c7d32cc41a","Type":"ContainerStarted","Data":"6f274359d7cd414342db203279a4a332acab0caf6b6f2479d75dfe007525bfae"}
Nov 04 03:39:09.049672 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:09.049636    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-26xvk" event={"ID":"621375f0-75dd-445b-8e50-50b52600a7dc","Type":"ContainerStarted","Data":"cc716d762db61e3b50ed2d96eb2c5243b127a10fb35ec5a05b9b9136b13d7003"}
Nov 04 03:39:09.049800 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:09.049688    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-26xvk" event={"ID":"621375f0-75dd-445b-8e50-50b52600a7dc","Type":"ContainerStarted","Data":"0331ce5bc759b0b1dd2612eb8108aba4a2421ab41fbf4b43bee3067ce1e7d9fc"}
Nov 04 03:39:09.317646 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:09.317529    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk" podStartSLOduration=2.317504426 podStartE2EDuration="2.317504426s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:39:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:39:09.165285493 +0000 UTC m=+1291.741938433" watchObservedRunningTime="2026-11-04 03:39:09.317504426 +0000 UTC m=+1291.894157341"
Nov 04 03:39:09.471095 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:09.471057    5609 patch_prober.go:29] interesting pod/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 04 03:39:09.471270 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:09.471123    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c" podUID="5e6b2f82-51f7-49e4-b952-7a0861e35889" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 04 03:39:09.893342 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:09.893304    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk"
Nov 04 03:39:09.893767 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:09.893742    5609 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Nov 04 03:39:09.893935 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:09.893912    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk" podUID="fd1ae838-b620-411d-bd66-82c7d32cc41a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Nov 04 03:39:09.917722 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:09.917683    5609 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-26xvk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body=
Nov 04 03:39:09.917974 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:09.917944    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-26xvk" podUID="621375f0-75dd-445b-8e50-50b52600a7dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused"
Nov 04 03:39:09.918152 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:09.918139    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-26xvk"
Nov 04 03:39:10.475326 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:10.474889    5609 patch_prober.go:29] interesting pod/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 04 03:39:10.475326 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:10.474959    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c" podUID="5e6b2f82-51f7-49e4-b952-7a0861e35889" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 04 03:39:10.909838 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:10.909294    5609 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Nov 04 03:39:10.909838 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:10.909354    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk" podUID="fd1ae838-b620-411d-bd66-82c7d32cc41a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Nov 04 03:39:10.923955 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:10.923920    5609 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-26xvk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body=
Nov 04 03:39:10.924220 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:10.924194    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-26xvk" podUID="621375f0-75dd-445b-8e50-50b52600a7dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused"
Nov 04 03:39:11.471171 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:11.471133    5609 patch_prober.go:29] interesting pod/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 04 03:39:11.471352 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:11.471197    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c" podUID="5e6b2f82-51f7-49e4-b952-7a0861e35889" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 04 03:39:11.906483 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:11.900898    5609 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Nov 04 03:39:11.906483 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:11.900965    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk" podUID="fd1ae838-b620-411d-bd66-82c7d32cc41a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Nov 04 03:39:11.934427 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:11.931004    5609 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-26xvk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body=
Nov 04 03:39:11.934427 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:11.931065    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-26xvk" podUID="621375f0-75dd-445b-8e50-50b52600a7dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused"
Nov 04 03:39:12.471425 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:12.470992    5609 patch_prober.go:29] interesting pod/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 04 03:39:12.471425 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:12.471069    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c" podUID="5e6b2f82-51f7-49e4-b952-7a0861e35889" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 04 03:39:12.894864 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:12.894772    5609 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Nov 04 03:39:12.895129 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:12.895093    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk" podUID="fd1ae838-b620-411d-bd66-82c7d32cc41a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Nov 04 03:39:12.915728 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:12.915693    5609 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-26xvk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body=
Nov 04 03:39:12.916283 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:12.916255    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-26xvk" podUID="621375f0-75dd-445b-8e50-50b52600a7dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused"
Nov 04 03:39:13.476496 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:13.476462    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c"
Nov 04 03:39:13.476953 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:13.476930    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c"
Nov 04 03:39:13.477731 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:13.477717    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c"
Nov 04 03:39:13.714296 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:13.714233    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-26xvk" podStartSLOduration=6.714212949 podStartE2EDuration="6.714212949s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:39:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:39:09.292727198 +0000 UTC m=+1291.869380113" watchObservedRunningTime="2026-11-04 03:39:13.714212949 +0000 UTC m=+1296.290865865"
Nov 04 03:39:13.924699 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:13.905364    5609 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Nov 04 03:39:13.925329 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:13.925290    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk" podUID="fd1ae838-b620-411d-bd66-82c7d32cc41a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Nov 04 03:39:13.926156 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:13.926132    5609 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-26xvk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body=
Nov 04 03:39:13.926320 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:13.926301    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-26xvk" podUID="621375f0-75dd-445b-8e50-50b52600a7dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused"
Nov 04 03:39:14.892687 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:14.892650    5609 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Nov 04 03:39:14.892906 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:14.892714    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk" podUID="fd1ae838-b620-411d-bd66-82c7d32cc41a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Nov 04 03:39:14.915038 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:14.915003    5609 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-26xvk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body=
Nov 04 03:39:14.915173 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:14.915067    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-26xvk" podUID="621375f0-75dd-445b-8e50-50b52600a7dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused"
Nov 04 03:39:15.891810 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:15.891769    5609 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Nov 04 03:39:15.892332 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:15.891832    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk" podUID="fd1ae838-b620-411d-bd66-82c7d32cc41a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Nov 04 03:39:15.915411 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:15.915361    5609 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-26xvk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body=
Nov 04 03:39:15.915580 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:15.915449    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-26xvk" podUID="621375f0-75dd-445b-8e50-50b52600a7dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused"
Nov 04 03:39:17.680227 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:17.680192    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk"
Nov 04 03:39:17.680746 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:17.680622    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk"
Nov 04 03:39:17.682443 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:17.682386    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-hgwbk"
Nov 04 03:39:17.709747 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:17.709720    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-26xvk"
Nov 04 03:39:17.710203 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:17.710186    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-26xvk"
Nov 04 03:39:17.711578 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:17.711555    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-26xvk"
Nov 04 03:39:18.561566 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:39:18.561535    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:39:42.782017 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:39:42.692367    5609 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.147s"
Nov 04 03:39:43.588423 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:43.527643    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-ee1947-74f4dc77d6-z7f9w"]
Nov 04 03:39:48.561566 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:39:48.561528    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:39:48.661186 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:48.661132    5609 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w1-ee1947-74f4dc77d6-z7f9w" podUID="ae56bf38-1c41-4c65-96c1-c723def99c82" containerName="payload-processing" containerID="cri-o://51e74f49acfa1da2c0e73b118d6ed1905e190ca76c4eb77010c26dff2d81e1ad" gracePeriod=25
Nov 04 03:39:50.458120 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:39:50.458084    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w1-ee1947-74f4dc77d6-z7f9w" podUID="ae56bf38-1c41-4c65-96c1-c723def99c82" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:40:00.456677 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:00.456643    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w1-ee1947-74f4dc77d6-z7f9w" podUID="ae56bf38-1c41-4c65-96c1-c723def99c82" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:40:08.947750 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:08.947727    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-ee1947-74f4dc77d6-z7f9w"
Nov 04 03:40:09.036134 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:09.036096    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ae56bf38-1c41-4c65-96c1-c723def99c82-kube-api-access-8k82m\" (UniqueName: \"kubernetes.io/projected/ae56bf38-1c41-4c65-96c1-c723def99c82-kube-api-access-8k82m\") pod \"ae56bf38-1c41-4c65-96c1-c723def99c82\" (UID: \"ae56bf38-1c41-4c65-96c1-c723def99c82\") "
Nov 04 03:40:09.036320 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:09.036225    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ae56bf38-1c41-4c65-96c1-c723def99c82-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae56bf38-1c41-4c65-96c1-c723def99c82-plugins-config-volume\") pod \"ae56bf38-1c41-4c65-96c1-c723def99c82\" (UID: \"ae56bf38-1c41-4c65-96c1-c723def99c82\") "
Nov 04 03:40:09.036697 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:09.036667    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae56bf38-1c41-4c65-96c1-c723def99c82-plugins-config-volume" pod "ae56bf38-1c41-4c65-96c1-c723def99c82" (UID: "ae56bf38-1c41-4c65-96c1-c723def99c82"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:40:09.036857 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:09.036837    5609 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae56bf38-1c41-4c65-96c1-c723def99c82-plugins-config-volume\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:09.038460 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:09.038431    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae56bf38-1c41-4c65-96c1-c723def99c82-kube-api-access-8k82m" pod "ae56bf38-1c41-4c65-96c1-c723def99c82" (UID: "ae56bf38-1c41-4c65-96c1-c723def99c82"). InnerVolumeSpecName "kube-api-access-8k82m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:40:09.121374 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:09.121339    5609 generic.go:356] "Generic (PLEG): container finished" podID="ae56bf38-1c41-4c65-96c1-c723def99c82" containerID="51e74f49acfa1da2c0e73b118d6ed1905e190ca76c4eb77010c26dff2d81e1ad" exitCode=0
Nov 04 03:40:09.121565 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:09.121390    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-ee1947-74f4dc77d6-z7f9w"
Nov 04 03:40:09.121565 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:09.121434    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-ee1947-74f4dc77d6-z7f9w" event={"ID":"ae56bf38-1c41-4c65-96c1-c723def99c82","Type":"ContainerDied","Data":"51e74f49acfa1da2c0e73b118d6ed1905e190ca76c4eb77010c26dff2d81e1ad"}
Nov 04 03:40:09.121565 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:09.121500    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-ee1947-74f4dc77d6-z7f9w" event={"ID":"ae56bf38-1c41-4c65-96c1-c723def99c82","Type":"ContainerDied","Data":"0a1f9b3ed39ccf9ca5f749c3714529e9223729fcba53530ecdf1afd272c1291c"}
Nov 04 03:40:09.121565 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:09.121530    5609 scope.go:122] "RemoveContainer" containerID="51e74f49acfa1da2c0e73b118d6ed1905e190ca76c4eb77010c26dff2d81e1ad"
Nov 04 03:40:09.134717 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:09.134696    5609 scope.go:122] "RemoveContainer" containerID="51e74f49acfa1da2c0e73b118d6ed1905e190ca76c4eb77010c26dff2d81e1ad"
Nov 04 03:40:09.135026 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:40:09.135002    5609 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51e74f49acfa1da2c0e73b118d6ed1905e190ca76c4eb77010c26dff2d81e1ad\": container with ID starting with 51e74f49acfa1da2c0e73b118d6ed1905e190ca76c4eb77010c26dff2d81e1ad not found: ID does not exist" containerID="51e74f49acfa1da2c0e73b118d6ed1905e190ca76c4eb77010c26dff2d81e1ad"
Nov 04 03:40:09.135149 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:09.135031    5609 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51e74f49acfa1da2c0e73b118d6ed1905e190ca76c4eb77010c26dff2d81e1ad"} err="failed to get container status \"51e74f49acfa1da2c0e73b118d6ed1905e190ca76c4eb77010c26dff2d81e1ad\": rpc error: code = NotFound desc = could not find container \"51e74f49acfa1da2c0e73b118d6ed1905e190ca76c4eb77010c26dff2d81e1ad\": container with ID starting with 51e74f49acfa1da2c0e73b118d6ed1905e190ca76c4eb77010c26dff2d81e1ad not found: ID does not exist"
Nov 04 03:40:09.137761 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:09.137741    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8k82m\" (UniqueName: \"kubernetes.io/projected/ae56bf38-1c41-4c65-96c1-c723def99c82-kube-api-access-8k82m\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:09.278957 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:09.278119    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-ee1947-74f4dc77d6-z7f9w"]
Nov 04 03:40:09.282238 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:09.282208    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-ee1947-74f4dc77d6-z7f9w"]
Nov 04 03:40:09.537108 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:09.537026    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ae56bf38-1c41-4c65-96c1-c723def99c82" path="/var/lib/kubelet/pods/ae56bf38-1c41-4c65-96c1-c723def99c82/volumes"
Nov 04 03:40:18.561005 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:40:18.560927    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:40:20.992991 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:20.992953    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s"]
Nov 04 03:40:20.993498 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:20.993225    5609 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s" podUID="1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99" containerName="istio-proxy" containerID="cri-o://9af6ef4cce350d3c87edfd993ebc5fe4a97c95e904e48f986790da22a526a49f" gracePeriod=30
Nov 04 03:40:21.037229 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:21.037189    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb"]
Nov 04 03:40:21.037630 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:21.037463    5609 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb" podUID="8003391e-2385-45df-b9cf-30d5c564a0c6" containerName="istio-proxy" containerID="cri-o://5a0dde2b6cbf34972d5639e23e4b21e46e2d6c59c076185c8c19f3876032a9a8" gracePeriod=30
Nov 04 03:40:21.040903 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:21.040875    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd"]
Nov 04 03:40:21.041091 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:21.041068    5609 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd" podUID="297439ad-252f-487e-a47b-3cf22cddb1b6" containerName="istio-proxy" containerID="cri-o://638f27e833bc1f91dd51e7442824a9efd31da0b2d25326765ffc7d0934a8cf81" gracePeriod=30
Nov 04 03:40:21.044243 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:21.044219    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw"]
Nov 04 03:40:21.044474 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:21.044453    5609 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw" podUID="7dd4f492-1db4-4368-b0b3-23661c65b6d2" containerName="istio-proxy" containerID="cri-o://7e46a85065f2720af67542ad6f53d5bdbe6713b35b67e9301882040a3f277aed" gracePeriod=30
Nov 04 03:40:22.100396 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:22.100354    5609 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 03:40:22.100830 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:22.100462    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd" podUID="297439ad-252f-487e-a47b-3cf22cddb1b6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 03:40:22.541942 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:22.541907    5609 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 03:40:22.542121 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:22.541969    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw" podUID="7dd4f492-1db4-4368-b0b3-23661c65b6d2" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 03:40:22.587583 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:22.587552    5609 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 03:40:22.587725 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:22.587607    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s" podUID="1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 03:40:25.979604 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:25.979568    5609 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 03:40:25.980007 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:25.979621    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb" podUID="8003391e-2385-45df-b9cf-30d5c564a0c6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 03:40:26.318315 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.318284    5609 generic.go:356] "Generic (PLEG): container finished" podID="1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99" containerID="9af6ef4cce350d3c87edfd993ebc5fe4a97c95e904e48f986790da22a526a49f" exitCode=0
Nov 04 03:40:26.318469 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.318385    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s" event={"ID":"1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99","Type":"ContainerDied","Data":"9af6ef4cce350d3c87edfd993ebc5fe4a97c95e904e48f986790da22a526a49f"}
Nov 04 03:40:26.320435 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.320326    5609 generic.go:356] "Generic (PLEG): container finished" podID="7dd4f492-1db4-4368-b0b3-23661c65b6d2" containerID="7e46a85065f2720af67542ad6f53d5bdbe6713b35b67e9301882040a3f277aed" exitCode=0
Nov 04 03:40:26.320435 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.320415    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw" event={"ID":"7dd4f492-1db4-4368-b0b3-23661c65b6d2","Type":"ContainerDied","Data":"7e46a85065f2720af67542ad6f53d5bdbe6713b35b67e9301882040a3f277aed"}
Nov 04 03:40:26.322564 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.322542    5609 generic.go:356] "Generic (PLEG): container finished" podID="8003391e-2385-45df-b9cf-30d5c564a0c6" containerID="5a0dde2b6cbf34972d5639e23e4b21e46e2d6c59c076185c8c19f3876032a9a8" exitCode=0
Nov 04 03:40:26.322665 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.322614    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb" event={"ID":"8003391e-2385-45df-b9cf-30d5c564a0c6","Type":"ContainerDied","Data":"5a0dde2b6cbf34972d5639e23e4b21e46e2d6c59c076185c8c19f3876032a9a8"}
Nov 04 03:40:26.322731 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.322714    5609 scope.go:122] "RemoveContainer" containerID="f17f6a6c1fa388ab7f94d92ad013bd9cd3d57fcb3fc464810251de0251554873"
Nov 04 03:40:26.325511 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.325486    5609 generic.go:356] "Generic (PLEG): container finished" podID="297439ad-252f-487e-a47b-3cf22cddb1b6" containerID="638f27e833bc1f91dd51e7442824a9efd31da0b2d25326765ffc7d0934a8cf81" exitCode=0
Nov 04 03:40:26.325696 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.325677    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd" event={"ID":"297439ad-252f-487e-a47b-3cf22cddb1b6","Type":"ContainerDied","Data":"638f27e833bc1f91dd51e7442824a9efd31da0b2d25326765ffc7d0934a8cf81"}
Nov 04 03:40:26.345960 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.345929    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb"
Nov 04 03:40:26.362606 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.362577    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd"
Nov 04 03:40:26.439644 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.439522    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8003391e-2385-45df-b9cf-30d5c564a0c6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8003391e-2385-45df-b9cf-30d5c564a0c6-credential-socket\") pod \"8003391e-2385-45df-b9cf-30d5c564a0c6\" (UID: \"8003391e-2385-45df-b9cf-30d5c564a0c6\") "
Nov 04 03:40:26.439644 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.439606    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/297439ad-252f-487e-a47b-3cf22cddb1b6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/297439ad-252f-487e-a47b-3cf22cddb1b6-istio-podinfo\") pod \"297439ad-252f-487e-a47b-3cf22cddb1b6\" (UID: \"297439ad-252f-487e-a47b-3cf22cddb1b6\") "
Nov 04 03:40:26.439780 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.439673    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8003391e-2385-45df-b9cf-30d5c564a0c6-kube-api-access-rc87z\" (UniqueName: \"kubernetes.io/projected/8003391e-2385-45df-b9cf-30d5c564a0c6-kube-api-access-rc87z\") pod \"8003391e-2385-45df-b9cf-30d5c564a0c6\" (UID: \"8003391e-2385-45df-b9cf-30d5c564a0c6\") "
Nov 04 03:40:26.439780 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.439705    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/297439ad-252f-487e-a47b-3cf22cddb1b6-istio-token\" (UniqueName: \"kubernetes.io/projected/297439ad-252f-487e-a47b-3cf22cddb1b6-istio-token\") pod \"297439ad-252f-487e-a47b-3cf22cddb1b6\" (UID: \"297439ad-252f-487e-a47b-3cf22cddb1b6\") "
Nov 04 03:40:26.439780 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.439737    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8003391e-2385-45df-b9cf-30d5c564a0c6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8003391e-2385-45df-b9cf-30d5c564a0c6-workload-certs\") pod \"8003391e-2385-45df-b9cf-30d5c564a0c6\" (UID: \"8003391e-2385-45df-b9cf-30d5c564a0c6\") "
Nov 04 03:40:26.439780 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.439765    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/297439ad-252f-487e-a47b-3cf22cddb1b6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/297439ad-252f-487e-a47b-3cf22cddb1b6-istiod-ca-cert\") pod \"297439ad-252f-487e-a47b-3cf22cddb1b6\" (UID: \"297439ad-252f-487e-a47b-3cf22cddb1b6\") "
Nov 04 03:40:26.439981 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.439792    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/297439ad-252f-487e-a47b-3cf22cddb1b6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/297439ad-252f-487e-a47b-3cf22cddb1b6-istio-envoy\") pod \"297439ad-252f-487e-a47b-3cf22cddb1b6\" (UID: \"297439ad-252f-487e-a47b-3cf22cddb1b6\") "
Nov 04 03:40:26.439981 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.439817    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8003391e-2385-45df-b9cf-30d5c564a0c6-credential-socket" pod "8003391e-2385-45df-b9cf-30d5c564a0c6" (UID: "8003391e-2385-45df-b9cf-30d5c564a0c6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.439981 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.439836    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/297439ad-252f-487e-a47b-3cf22cddb1b6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/297439ad-252f-487e-a47b-3cf22cddb1b6-workload-socket\") pod \"297439ad-252f-487e-a47b-3cf22cddb1b6\" (UID: \"297439ad-252f-487e-a47b-3cf22cddb1b6\") "
Nov 04 03:40:26.439981 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.439877    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8003391e-2385-45df-b9cf-30d5c564a0c6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8003391e-2385-45df-b9cf-30d5c564a0c6-istiod-ca-cert\") pod \"8003391e-2385-45df-b9cf-30d5c564a0c6\" (UID: \"8003391e-2385-45df-b9cf-30d5c564a0c6\") "
Nov 04 03:40:26.439981 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.439915    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8003391e-2385-45df-b9cf-30d5c564a0c6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8003391e-2385-45df-b9cf-30d5c564a0c6-istio-envoy\") pod \"8003391e-2385-45df-b9cf-30d5c564a0c6\" (UID: \"8003391e-2385-45df-b9cf-30d5c564a0c6\") "
Nov 04 03:40:26.439981 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.439964    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8003391e-2385-45df-b9cf-30d5c564a0c6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8003391e-2385-45df-b9cf-30d5c564a0c6-istio-data\") pod \"8003391e-2385-45df-b9cf-30d5c564a0c6\" (UID: \"8003391e-2385-45df-b9cf-30d5c564a0c6\") "
Nov 04 03:40:26.440360 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.440057    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8003391e-2385-45df-b9cf-30d5c564a0c6-istio-token\" (UniqueName: \"kubernetes.io/projected/8003391e-2385-45df-b9cf-30d5c564a0c6-istio-token\") pod \"8003391e-2385-45df-b9cf-30d5c564a0c6\" (UID: \"8003391e-2385-45df-b9cf-30d5c564a0c6\") "
Nov 04 03:40:26.440360 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.440110    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/297439ad-252f-487e-a47b-3cf22cddb1b6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/297439ad-252f-487e-a47b-3cf22cddb1b6-credential-socket\") pod \"297439ad-252f-487e-a47b-3cf22cddb1b6\" (UID: \"297439ad-252f-487e-a47b-3cf22cddb1b6\") "
Nov 04 03:40:26.440360 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.440161    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8003391e-2385-45df-b9cf-30d5c564a0c6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8003391e-2385-45df-b9cf-30d5c564a0c6-workload-socket\") pod \"8003391e-2385-45df-b9cf-30d5c564a0c6\" (UID: \"8003391e-2385-45df-b9cf-30d5c564a0c6\") "
Nov 04 03:40:26.440360 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.440253    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8003391e-2385-45df-b9cf-30d5c564a0c6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8003391e-2385-45df-b9cf-30d5c564a0c6-istio-podinfo\") pod \"8003391e-2385-45df-b9cf-30d5c564a0c6\" (UID: \"8003391e-2385-45df-b9cf-30d5c564a0c6\") "
Nov 04 03:40:26.440360 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.440289    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/297439ad-252f-487e-a47b-3cf22cddb1b6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/297439ad-252f-487e-a47b-3cf22cddb1b6-istio-data\") pod \"297439ad-252f-487e-a47b-3cf22cddb1b6\" (UID: \"297439ad-252f-487e-a47b-3cf22cddb1b6\") "
Nov 04 03:40:26.440360 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.440314    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/297439ad-252f-487e-a47b-3cf22cddb1b6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/297439ad-252f-487e-a47b-3cf22cddb1b6-workload-certs\") pod \"297439ad-252f-487e-a47b-3cf22cddb1b6\" (UID: \"297439ad-252f-487e-a47b-3cf22cddb1b6\") "
Nov 04 03:40:26.441180 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.440365    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/297439ad-252f-487e-a47b-3cf22cddb1b6-kube-api-access-dhwxc\" (UniqueName: \"kubernetes.io/projected/297439ad-252f-487e-a47b-3cf22cddb1b6-kube-api-access-dhwxc\") pod \"297439ad-252f-487e-a47b-3cf22cddb1b6\" (UID: \"297439ad-252f-487e-a47b-3cf22cddb1b6\") "
Nov 04 03:40:26.442802 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.441453    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8003391e-2385-45df-b9cf-30d5c564a0c6-workload-certs" pod "8003391e-2385-45df-b9cf-30d5c564a0c6" (UID: "8003391e-2385-45df-b9cf-30d5c564a0c6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.442802 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.441792    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/297439ad-252f-487e-a47b-3cf22cddb1b6-istiod-ca-cert" pod "297439ad-252f-487e-a47b-3cf22cddb1b6" (UID: "297439ad-252f-487e-a47b-3cf22cddb1b6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:40:26.442802 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.442134    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8003391e-2385-45df-b9cf-30d5c564a0c6-istiod-ca-cert" pod "8003391e-2385-45df-b9cf-30d5c564a0c6" (UID: "8003391e-2385-45df-b9cf-30d5c564a0c6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:40:26.442802 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.442449    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/297439ad-252f-487e-a47b-3cf22cddb1b6-istio-token" pod "297439ad-252f-487e-a47b-3cf22cddb1b6" (UID: "297439ad-252f-487e-a47b-3cf22cddb1b6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:40:26.442802 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.442491    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/297439ad-252f-487e-a47b-3cf22cddb1b6-workload-socket" pod "297439ad-252f-487e-a47b-3cf22cddb1b6" (UID: "297439ad-252f-487e-a47b-3cf22cddb1b6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.444161 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.443206    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8003391e-2385-45df-b9cf-30d5c564a0c6-istio-token" pod "8003391e-2385-45df-b9cf-30d5c564a0c6" (UID: "8003391e-2385-45df-b9cf-30d5c564a0c6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:40:26.444161 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.443429    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/297439ad-252f-487e-a47b-3cf22cddb1b6-istio-envoy" pod "297439ad-252f-487e-a47b-3cf22cddb1b6" (UID: "297439ad-252f-487e-a47b-3cf22cddb1b6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.444161 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.443465    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8003391e-2385-45df-b9cf-30d5c564a0c6-istio-data" pod "8003391e-2385-45df-b9cf-30d5c564a0c6" (UID: "8003391e-2385-45df-b9cf-30d5c564a0c6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.444161 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.443644    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/297439ad-252f-487e-a47b-3cf22cddb1b6-istio-data" pod "297439ad-252f-487e-a47b-3cf22cddb1b6" (UID: "297439ad-252f-487e-a47b-3cf22cddb1b6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.444161 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.443799    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/297439ad-252f-487e-a47b-3cf22cddb1b6-workload-certs" pod "297439ad-252f-487e-a47b-3cf22cddb1b6" (UID: "297439ad-252f-487e-a47b-3cf22cddb1b6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.444161 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.443959    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/297439ad-252f-487e-a47b-3cf22cddb1b6-credential-socket" pod "297439ad-252f-487e-a47b-3cf22cddb1b6" (UID: "297439ad-252f-487e-a47b-3cf22cddb1b6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.444161 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.444131    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8003391e-2385-45df-b9cf-30d5c564a0c6-workload-socket" pod "8003391e-2385-45df-b9cf-30d5c564a0c6" (UID: "8003391e-2385-45df-b9cf-30d5c564a0c6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.445517 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.445470    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8003391e-2385-45df-b9cf-30d5c564a0c6-kube-api-access-rc87z" pod "8003391e-2385-45df-b9cf-30d5c564a0c6" (UID: "8003391e-2385-45df-b9cf-30d5c564a0c6"). InnerVolumeSpecName "kube-api-access-rc87z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:40:26.446285 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.446208    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw"
Nov 04 03:40:26.446609 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.446588    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rc87z\" (UniqueName: \"kubernetes.io/projected/8003391e-2385-45df-b9cf-30d5c564a0c6-kube-api-access-rc87z\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.446696 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.446614    5609 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/297439ad-252f-487e-a47b-3cf22cddb1b6-istio-token\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.446696 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.446628    5609 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8003391e-2385-45df-b9cf-30d5c564a0c6-workload-certs\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.446696 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.446643    5609 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/297439ad-252f-487e-a47b-3cf22cddb1b6-istiod-ca-cert\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.446696 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.446656    5609 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/297439ad-252f-487e-a47b-3cf22cddb1b6-istio-envoy\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.446696 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.446669    5609 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/297439ad-252f-487e-a47b-3cf22cddb1b6-workload-socket\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.446696 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.446682    5609 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8003391e-2385-45df-b9cf-30d5c564a0c6-istiod-ca-cert\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.446696 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.446695    5609 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8003391e-2385-45df-b9cf-30d5c564a0c6-istio-data\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.446945 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.446708    5609 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8003391e-2385-45df-b9cf-30d5c564a0c6-istio-token\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.446945 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.446721    5609 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/297439ad-252f-487e-a47b-3cf22cddb1b6-credential-socket\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.446945 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.446735    5609 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8003391e-2385-45df-b9cf-30d5c564a0c6-workload-socket\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.446945 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.446748    5609 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/297439ad-252f-487e-a47b-3cf22cddb1b6-istio-data\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.446945 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.446763    5609 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/297439ad-252f-487e-a47b-3cf22cddb1b6-workload-certs\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.446945 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.446779    5609 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8003391e-2385-45df-b9cf-30d5c564a0c6-credential-socket\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.447709 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.447686    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/297439ad-252f-487e-a47b-3cf22cddb1b6-kube-api-access-dhwxc" pod "297439ad-252f-487e-a47b-3cf22cddb1b6" (UID: "297439ad-252f-487e-a47b-3cf22cddb1b6"). InnerVolumeSpecName "kube-api-access-dhwxc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:40:26.449375 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.449193    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8003391e-2385-45df-b9cf-30d5c564a0c6-istio-podinfo" pod "8003391e-2385-45df-b9cf-30d5c564a0c6" (UID: "8003391e-2385-45df-b9cf-30d5c564a0c6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:40:26.450557 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.450536    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8003391e-2385-45df-b9cf-30d5c564a0c6-istio-envoy" pod "8003391e-2385-45df-b9cf-30d5c564a0c6" (UID: "8003391e-2385-45df-b9cf-30d5c564a0c6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.450865 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.450845    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/297439ad-252f-487e-a47b-3cf22cddb1b6-istio-podinfo" pod "297439ad-252f-487e-a47b-3cf22cddb1b6" (UID: "297439ad-252f-487e-a47b-3cf22cddb1b6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:40:26.454031 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.454008    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s"
Nov 04 03:40:26.547961 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.547923    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7dd4f492-1db4-4368-b0b3-23661c65b6d2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dd4f492-1db4-4368-b0b3-23661c65b6d2-workload-socket\") pod \"7dd4f492-1db4-4368-b0b3-23661c65b6d2\" (UID: \"7dd4f492-1db4-4368-b0b3-23661c65b6d2\") "
Nov 04 03:40:26.548112 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.548017    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7dd4f492-1db4-4368-b0b3-23661c65b6d2-istio-token\" (UniqueName: \"kubernetes.io/projected/7dd4f492-1db4-4368-b0b3-23661c65b6d2-istio-token\") pod \"7dd4f492-1db4-4368-b0b3-23661c65b6d2\" (UID: \"7dd4f492-1db4-4368-b0b3-23661c65b6d2\") "
Nov 04 03:40:26.548112 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.548050    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-istiod-ca-cert\") pod \"1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99\" (UID: \"1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99\") "
Nov 04 03:40:26.548112 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.548084    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-istio-envoy\") pod \"1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99\" (UID: \"1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99\") "
Nov 04 03:40:26.548296 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.548216    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-workload-certs\") pod \"1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99\" (UID: \"1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99\") "
Nov 04 03:40:26.548296 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.548235    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dd4f492-1db4-4368-b0b3-23661c65b6d2-workload-socket" pod "7dd4f492-1db4-4368-b0b3-23661c65b6d2" (UID: "7dd4f492-1db4-4368-b0b3-23661c65b6d2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.548451 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.548303    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7dd4f492-1db4-4368-b0b3-23661c65b6d2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7dd4f492-1db4-4368-b0b3-23661c65b6d2-istio-data\") pod \"7dd4f492-1db4-4368-b0b3-23661c65b6d2\" (UID: \"7dd4f492-1db4-4368-b0b3-23661c65b6d2\") "
Nov 04 03:40:26.548451 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.548381    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-workload-socket\") pod \"1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99\" (UID: \"1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99\") "
Nov 04 03:40:26.548564 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.548478    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7dd4f492-1db4-4368-b0b3-23661c65b6d2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7dd4f492-1db4-4368-b0b3-23661c65b6d2-istio-podinfo\") pod \"7dd4f492-1db4-4368-b0b3-23661c65b6d2\" (UID: \"7dd4f492-1db4-4368-b0b3-23661c65b6d2\") "
Nov 04 03:40:26.548564 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.548507    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-istiod-ca-cert" pod "1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99" (UID: "1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:40:26.548564 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.548537    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7dd4f492-1db4-4368-b0b3-23661c65b6d2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7dd4f492-1db4-4368-b0b3-23661c65b6d2-istiod-ca-cert\") pod \"7dd4f492-1db4-4368-b0b3-23661c65b6d2\" (UID: \"7dd4f492-1db4-4368-b0b3-23661c65b6d2\") "
Nov 04 03:40:26.548728 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.548572    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7dd4f492-1db4-4368-b0b3-23661c65b6d2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dd4f492-1db4-4368-b0b3-23661c65b6d2-credential-socket\") pod \"7dd4f492-1db4-4368-b0b3-23661c65b6d2\" (UID: \"7dd4f492-1db4-4368-b0b3-23661c65b6d2\") "
Nov 04 03:40:26.548728 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.548609    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-credential-socket\") pod \"1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99\" (UID: \"1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99\") "
Nov 04 03:40:26.548728 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.548650    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7dd4f492-1db4-4368-b0b3-23661c65b6d2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7dd4f492-1db4-4368-b0b3-23661c65b6d2-istio-envoy\") pod \"7dd4f492-1db4-4368-b0b3-23661c65b6d2\" (UID: \"7dd4f492-1db4-4368-b0b3-23661c65b6d2\") "
Nov 04 03:40:26.548728 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.548654    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dd4f492-1db4-4368-b0b3-23661c65b6d2-istio-data" pod "7dd4f492-1db4-4368-b0b3-23661c65b6d2" (UID: "7dd4f492-1db4-4368-b0b3-23661c65b6d2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.549138 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.548970    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7dd4f492-1db4-4368-b0b3-23661c65b6d2-istiod-ca-cert" pod "7dd4f492-1db4-4368-b0b3-23661c65b6d2" (UID: "7dd4f492-1db4-4368-b0b3-23661c65b6d2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:40:26.549138 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.549003    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-workload-certs" pod "1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99" (UID: "1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.549138 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.549090    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-workload-socket" pod "1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99" (UID: "1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.549349 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.549245    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dd4f492-1db4-4368-b0b3-23661c65b6d2-credential-socket" pod "7dd4f492-1db4-4368-b0b3-23661c65b6d2" (UID: "7dd4f492-1db4-4368-b0b3-23661c65b6d2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.549349 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.549260    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-credential-socket" pod "1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99" (UID: "1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.549472 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.549364    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-istio-podinfo\") pod \"1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99\" (UID: \"1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99\") "
Nov 04 03:40:26.549472 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.549415    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-istio-data\") pod \"1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99\" (UID: \"1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99\") "
Nov 04 03:40:26.549472 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.549457    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7dd4f492-1db4-4368-b0b3-23661c65b6d2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7dd4f492-1db4-4368-b0b3-23661c65b6d2-workload-certs\") pod \"7dd4f492-1db4-4368-b0b3-23661c65b6d2\" (UID: \"7dd4f492-1db4-4368-b0b3-23661c65b6d2\") "
Nov 04 03:40:26.549620 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.549512    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-kube-api-access-k95wm\" (UniqueName: \"kubernetes.io/projected/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-kube-api-access-k95wm\") pod \"1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99\" (UID: \"1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99\") "
Nov 04 03:40:26.549620 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.549540    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-istio-token\" (UniqueName: \"kubernetes.io/projected/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-istio-token\") pod \"1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99\" (UID: \"1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99\") "
Nov 04 03:40:26.549620 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.549575    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7dd4f492-1db4-4368-b0b3-23661c65b6d2-kube-api-access-qbg67\" (UniqueName: \"kubernetes.io/projected/7dd4f492-1db4-4368-b0b3-23661c65b6d2-kube-api-access-qbg67\") pod \"7dd4f492-1db4-4368-b0b3-23661c65b6d2\" (UID: \"7dd4f492-1db4-4368-b0b3-23661c65b6d2\") "
Nov 04 03:40:26.550744 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.550023    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dd4f492-1db4-4368-b0b3-23661c65b6d2-workload-certs" pod "7dd4f492-1db4-4368-b0b3-23661c65b6d2" (UID: "7dd4f492-1db4-4368-b0b3-23661c65b6d2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.550744 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.550281    5609 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7dd4f492-1db4-4368-b0b3-23661c65b6d2-istiod-ca-cert\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.550744 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.550306    5609 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dd4f492-1db4-4368-b0b3-23661c65b6d2-credential-socket\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.550744 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.550323    5609 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-credential-socket\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.550744 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.550339    5609 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7dd4f492-1db4-4368-b0b3-23661c65b6d2-workload-certs\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.550744 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.550352    5609 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8003391e-2385-45df-b9cf-30d5c564a0c6-istio-podinfo\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.550744 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.550364    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dhwxc\" (UniqueName: \"kubernetes.io/projected/297439ad-252f-487e-a47b-3cf22cddb1b6-kube-api-access-dhwxc\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.550744 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.550378    5609 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dd4f492-1db4-4368-b0b3-23661c65b6d2-workload-socket\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.550744 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.550391    5609 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/297439ad-252f-487e-a47b-3cf22cddb1b6-istio-podinfo\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.550744 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.550442    5609 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-istiod-ca-cert\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.550744 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.550456    5609 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-workload-certs\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.550744 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.550470    5609 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7dd4f492-1db4-4368-b0b3-23661c65b6d2-istio-data\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.550744 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.550486    5609 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-workload-socket\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.550744 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.550500    5609 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8003391e-2385-45df-b9cf-30d5c564a0c6-istio-envoy\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.550744 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.550625    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-istio-envoy" pod "1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99" (UID: "1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.551674 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.550860    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-istio-data" pod "1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99" (UID: "1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.551674 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.550976    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7dd4f492-1db4-4368-b0b3-23661c65b6d2-istio-token" pod "7dd4f492-1db4-4368-b0b3-23661c65b6d2" (UID: "7dd4f492-1db4-4368-b0b3-23661c65b6d2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:40:26.551674 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.551620    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dd4f492-1db4-4368-b0b3-23661c65b6d2-istio-envoy" pod "7dd4f492-1db4-4368-b0b3-23661c65b6d2" (UID: "7dd4f492-1db4-4368-b0b3-23661c65b6d2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.551846 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.551767    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7dd4f492-1db4-4368-b0b3-23661c65b6d2-istio-podinfo" pod "7dd4f492-1db4-4368-b0b3-23661c65b6d2" (UID: "7dd4f492-1db4-4368-b0b3-23661c65b6d2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:40:26.552546 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.552517    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-kube-api-access-k95wm" pod "1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99" (UID: "1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99"). InnerVolumeSpecName "kube-api-access-k95wm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:40:26.552873 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.552843    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7dd4f492-1db4-4368-b0b3-23661c65b6d2-kube-api-access-qbg67" pod "7dd4f492-1db4-4368-b0b3-23661c65b6d2" (UID: "7dd4f492-1db4-4368-b0b3-23661c65b6d2"). InnerVolumeSpecName "kube-api-access-qbg67". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:40:26.552995 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.552977    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-istio-token" pod "1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99" (UID: "1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:40:26.553214 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.553192    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-istio-podinfo" pod "1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99" (UID: "1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:40:26.652055 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.652024    5609 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7dd4f492-1db4-4368-b0b3-23661c65b6d2-istio-podinfo\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.652055 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.652051    5609 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7dd4f492-1db4-4368-b0b3-23661c65b6d2-istio-envoy\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.652055 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.652061    5609 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-istio-podinfo\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.652272 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.652069    5609 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-istio-data\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.652272 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.652078    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k95wm\" (UniqueName: \"kubernetes.io/projected/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-kube-api-access-k95wm\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.652272 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.652088    5609 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-istio-token\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.652272 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.652096    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qbg67\" (UniqueName: \"kubernetes.io/projected/7dd4f492-1db4-4368-b0b3-23661c65b6d2-kube-api-access-qbg67\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.652272 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.652104    5609 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7dd4f492-1db4-4368-b0b3-23661c65b6d2-istio-token\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.652272 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:26.652112    5609 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99-istio-envoy\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:27.391474 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:27.386632    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s" event={"ID":"1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99","Type":"ContainerDied","Data":"b1e72012fa50d6db2486a13c65b5025f51dc30ecef8de136c7066769d2d55bd5"}
Nov 04 03:40:27.391474 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:27.386705    5609 scope.go:122] "RemoveContainer" containerID="9af6ef4cce350d3c87edfd993ebc5fe4a97c95e904e48f986790da22a526a49f"
Nov 04 03:40:27.391474 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:27.386825    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s"
Nov 04 03:40:27.410690 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:27.409904    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw" event={"ID":"7dd4f492-1db4-4368-b0b3-23661c65b6d2","Type":"ContainerDied","Data":"49411c703a4e2c63b2fb2c644f4044d39ea2ced742b35425aa211892b3d078a2"}
Nov 04 03:40:27.410690 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:27.410026    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw"
Nov 04 03:40:27.428431 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:27.424486    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb"
Nov 04 03:40:27.428627 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:27.428524    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb" event={"ID":"8003391e-2385-45df-b9cf-30d5c564a0c6","Type":"ContainerDied","Data":"5129b03e2db47aaf16dbc7ba6fa1f67a67e7ab2302b8105feda5bf36591c5934"}
Nov 04 03:40:27.435684 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:27.435654    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd" event={"ID":"297439ad-252f-487e-a47b-3cf22cddb1b6","Type":"ContainerDied","Data":"6d5047a59560311ff14b0da2d5d54b5b35c43d3833425b78e67bdf4e7799a7e3"}
Nov 04 03:40:27.435810 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:27.435741    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd"
Nov 04 03:40:27.449495 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:27.449471    5609 scope.go:122] "RemoveContainer" containerID="7e46a85065f2720af67542ad6f53d5bdbe6713b35b67e9301882040a3f277aed"
Nov 04 03:40:27.460374 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:27.460355    5609 scope.go:122] "RemoveContainer" containerID="5a0dde2b6cbf34972d5639e23e4b21e46e2d6c59c076185c8c19f3876032a9a8"
Nov 04 03:40:27.470882 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:27.470855    5609 scope.go:122] "RemoveContainer" containerID="638f27e833bc1f91dd51e7442824a9efd31da0b2d25326765ffc7d0934a8cf81"
Nov 04 03:40:27.725603 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:27.725564    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw"]
Nov 04 03:40:27.728148 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:27.728124    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-llmgw"]
Nov 04 03:40:27.835843 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:27.835811    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd"]
Nov 04 03:40:27.841810 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:27.841785    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-5jkwd"]
Nov 04 03:40:27.947679 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:27.947646    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb"]
Nov 04 03:40:27.955666 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:27.955624    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-549gb"]
Nov 04 03:40:28.056093 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:28.055980    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s"]
Nov 04 03:40:28.059577 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:28.059555    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-zlp2s"]
Nov 04 03:40:29.534961 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:29.534931    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99" path="/var/lib/kubelet/pods/1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99/volumes"
Nov 04 03:40:29.535447 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:29.535428    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="297439ad-252f-487e-a47b-3cf22cddb1b6" path="/var/lib/kubelet/pods/297439ad-252f-487e-a47b-3cf22cddb1b6/volumes"
Nov 04 03:40:29.535875 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:29.535862    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7dd4f492-1db4-4368-b0b3-23661c65b6d2" path="/var/lib/kubelet/pods/7dd4f492-1db4-4368-b0b3-23661c65b6d2/volumes"
Nov 04 03:40:29.536589 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:40:29.536575    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8003391e-2385-45df-b9cf-30d5c564a0c6" path="/var/lib/kubelet/pods/8003391e-2385-45df-b9cf-30d5c564a0c6/volumes"
Nov 04 03:40:48.561015 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:40:48.560981    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:41:18.561383 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:41:18.561345    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:41:28.608761 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:28.608726    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-8ecbc5-776655854d-p7qp2"]
Nov 04 03:41:28.711612 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:28.711568    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-8ecbc5-7df459b5c9-f2hdp"]
Nov 04 03:41:33.610018 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:33.609958    5609 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-a-8ecbc5-776655854d-p7qp2" podUID="f4096fc6-30ab-48db-82f5-85f9847473e0" containerName="payload-processing" containerID="cri-o://1214221dd74bd9b042a302adf89e3cac1c8dcd8655770aea6f5e04515de85e17" gracePeriod=25
Nov 04 03:41:33.712831 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:33.712741    5609 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-8ecbc5-7df459b5c9-f2hdp" podUID="b263e963-52bb-4426-b6e6-e0b18f1ee810" containerName="payload-pre-processing" containerID="cri-o://8f914a65668790af43405e0226c3985efd8914101b4b46193e7ab513e6b79e26" gracePeriod=25
Nov 04 03:41:37.896201 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:37.896161    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-8ecbc5-59968fdc4b-lsf4c"]
Nov 04 03:41:37.896661 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:37.896482    5609 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-8ecbc5-59968fdc4b-lsf4c" podUID="d7610c80-027e-4494-8f93-2d275c06c796" containerName="maas-api" containerID="cri-o://00fd37b445a5310bdfc77be17926dfa416526a53a359b391d838ed56e2629449" gracePeriod=30
Nov 04 03:41:38.017756 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:38.017718    5609 generic.go:356] "Generic (PLEG): container finished" podID="d7610c80-027e-4494-8f93-2d275c06c796" containerID="00fd37b445a5310bdfc77be17926dfa416526a53a359b391d838ed56e2629449" exitCode=0
Nov 04 03:41:38.017926 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:38.017812    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-8ecbc5-59968fdc4b-lsf4c" event={"ID":"d7610c80-027e-4494-8f93-2d275c06c796","Type":"ContainerDied","Data":"00fd37b445a5310bdfc77be17926dfa416526a53a359b391d838ed56e2629449"}
Nov 04 03:41:38.222637 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:38.222612    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-8ecbc5-59968fdc4b-lsf4c"
Nov 04 03:41:38.332910 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:38.331890    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d7610c80-027e-4494-8f93-2d275c06c796-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7610c80-027e-4494-8f93-2d275c06c796-maas-api-tls\") pod \"d7610c80-027e-4494-8f93-2d275c06c796\" (UID: \"d7610c80-027e-4494-8f93-2d275c06c796\") "
Nov 04 03:41:38.334246 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:38.333170    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d7610c80-027e-4494-8f93-2d275c06c796-metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7610c80-027e-4494-8f93-2d275c06c796-metrics-certs\") pod \"d7610c80-027e-4494-8f93-2d275c06c796\" (UID: \"d7610c80-027e-4494-8f93-2d275c06c796\") "
Nov 04 03:41:38.334246 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:38.333247    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d7610c80-027e-4494-8f93-2d275c06c796-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7610c80-027e-4494-8f93-2d275c06c796-openshift-service-ca\") pod \"d7610c80-027e-4494-8f93-2d275c06c796\" (UID: \"d7610c80-027e-4494-8f93-2d275c06c796\") "
Nov 04 03:41:38.334246 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:38.333301    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d7610c80-027e-4494-8f93-2d275c06c796-kube-api-access-9sg76\" (UniqueName: \"kubernetes.io/projected/d7610c80-027e-4494-8f93-2d275c06c796-kube-api-access-9sg76\") pod \"d7610c80-027e-4494-8f93-2d275c06c796\" (UID: \"d7610c80-027e-4494-8f93-2d275c06c796\") "
Nov 04 03:41:38.335660 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:38.335334    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7610c80-027e-4494-8f93-2d275c06c796-openshift-service-ca" pod "d7610c80-027e-4494-8f93-2d275c06c796" (UID: "d7610c80-027e-4494-8f93-2d275c06c796"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:41:38.383616 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:38.383361    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7610c80-027e-4494-8f93-2d275c06c796-kube-api-access-9sg76" pod "d7610c80-027e-4494-8f93-2d275c06c796" (UID: "d7610c80-027e-4494-8f93-2d275c06c796"). InnerVolumeSpecName "kube-api-access-9sg76". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:41:38.392213 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:38.390049    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7610c80-027e-4494-8f93-2d275c06c796-maas-api-tls" pod "d7610c80-027e-4494-8f93-2d275c06c796" (UID: "d7610c80-027e-4494-8f93-2d275c06c796"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:41:38.392213 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:38.391315    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7610c80-027e-4494-8f93-2d275c06c796-metrics-certs" pod "d7610c80-027e-4494-8f93-2d275c06c796" (UID: "d7610c80-027e-4494-8f93-2d275c06c796"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:41:38.435042 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:38.435008    5609 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7610c80-027e-4494-8f93-2d275c06c796-maas-api-tls\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:41:38.435042 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:38.435038    5609 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7610c80-027e-4494-8f93-2d275c06c796-metrics-certs\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:41:38.435042 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:38.435049    5609 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7610c80-027e-4494-8f93-2d275c06c796-openshift-service-ca\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:41:38.435305 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:38.435058    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9sg76\" (UniqueName: \"kubernetes.io/projected/d7610c80-027e-4494-8f93-2d275c06c796-kube-api-access-9sg76\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:41:39.025949 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:39.025919    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-8ecbc5-59968fdc4b-lsf4c"
Nov 04 03:41:39.025949 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:39.025942    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-8ecbc5-59968fdc4b-lsf4c" event={"ID":"d7610c80-027e-4494-8f93-2d275c06c796","Type":"ContainerDied","Data":"6b26a9aa44d61f36dc303864ff67bb2c286cb2ccea776473b8987caa80194ebf"}
Nov 04 03:41:39.026480 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:39.026005    5609 scope.go:122] "RemoveContainer" containerID="00fd37b445a5310bdfc77be17926dfa416526a53a359b391d838ed56e2629449"
Nov 04 03:41:39.184352 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:39.184316    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-8ecbc5-59968fdc4b-lsf4c"]
Nov 04 03:41:39.231078 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:39.231043    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-8ecbc5-59968fdc4b-lsf4c"]
Nov 04 03:41:39.535822 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:39.535790    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d7610c80-027e-4494-8f93-2d275c06c796" path="/var/lib/kubelet/pods/d7610c80-027e-4494-8f93-2d275c06c796/volumes"
Nov 04 03:41:39.795369 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:39.795289    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-8ecbc5-7df459b5c9-f2hdp" podUID="b263e963-52bb-4426-b6e6-e0b18f1ee810" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:41:39.795532 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:39.795296    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-a-8ecbc5-776655854d-p7qp2" podUID="f4096fc6-30ab-48db-82f5-85f9847473e0" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:41:45.057335 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:45.057302    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v"]
Nov 04 03:41:45.057761 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:45.057520    5609 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v" podUID="20f015bc-d781-4317-99c3-ce697c740441" containerName="istio-proxy" containerID="cri-o://afab82e4b59537fe6181b47d2e50b98096f67715d10b0eeb13141eece72da517" gracePeriod=30
Nov 04 03:41:46.806484 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:46.806450    5609 patch_prober.go:29] interesting pod/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 03:41:46.806872 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:46.806511    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v" podUID="20f015bc-d781-4317-99c3-ce697c740441" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 03:41:48.559927 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:41:48.559843    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:41:49.794620 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:49.794578    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-8ecbc5-7df459b5c9-f2hdp" podUID="b263e963-52bb-4426-b6e6-e0b18f1ee810" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:41:49.794620 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:49.794603    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-a-8ecbc5-776655854d-p7qp2" podUID="f4096fc6-30ab-48db-82f5-85f9847473e0" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:41:50.328627 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:50.328558    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v"
Nov 04 03:41:50.430104 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:50.430027    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/20f015bc-d781-4317-99c3-ce697c740441-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20f015bc-d781-4317-99c3-ce697c740441-istiod-ca-cert\") pod \"20f015bc-d781-4317-99c3-ce697c740441\" (UID: \"20f015bc-d781-4317-99c3-ce697c740441\") "
Nov 04 03:41:50.430291 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:50.430124    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20f015bc-d781-4317-99c3-ce697c740441-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20f015bc-d781-4317-99c3-ce697c740441-workload-certs\") pod \"20f015bc-d781-4317-99c3-ce697c740441\" (UID: \"20f015bc-d781-4317-99c3-ce697c740441\") "
Nov 04 03:41:50.430291 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:50.430183    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20f015bc-d781-4317-99c3-ce697c740441-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20f015bc-d781-4317-99c3-ce697c740441-istio-envoy\") pod \"20f015bc-d781-4317-99c3-ce697c740441\" (UID: \"20f015bc-d781-4317-99c3-ce697c740441\") "
Nov 04 03:41:50.430291 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:50.430245    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20f015bc-d781-4317-99c3-ce697c740441-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20f015bc-d781-4317-99c3-ce697c740441-credential-socket\") pod \"20f015bc-d781-4317-99c3-ce697c740441\" (UID: \"20f015bc-d781-4317-99c3-ce697c740441\") "
Nov 04 03:41:50.430291 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:50.430286    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20f015bc-d781-4317-99c3-ce697c740441-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20f015bc-d781-4317-99c3-ce697c740441-workload-socket\") pod \"20f015bc-d781-4317-99c3-ce697c740441\" (UID: \"20f015bc-d781-4317-99c3-ce697c740441\") "
Nov 04 03:41:50.430530 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:50.430332    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/20f015bc-d781-4317-99c3-ce697c740441-istio-token\" (UniqueName: \"kubernetes.io/projected/20f015bc-d781-4317-99c3-ce697c740441-istio-token\") pod \"20f015bc-d781-4317-99c3-ce697c740441\" (UID: \"20f015bc-d781-4317-99c3-ce697c740441\") "
Nov 04 03:41:50.430530 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:50.430420    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20f015bc-d781-4317-99c3-ce697c740441-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20f015bc-d781-4317-99c3-ce697c740441-istio-data\") pod \"20f015bc-d781-4317-99c3-ce697c740441\" (UID: \"20f015bc-d781-4317-99c3-ce697c740441\") "
Nov 04 03:41:50.430530 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:50.430434    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20f015bc-d781-4317-99c3-ce697c740441-istiod-ca-cert" pod "20f015bc-d781-4317-99c3-ce697c740441" (UID: "20f015bc-d781-4317-99c3-ce697c740441"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:41:50.430530 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:50.430501    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/20f015bc-d781-4317-99c3-ce697c740441-kube-api-access-kpdmq\" (UniqueName: \"kubernetes.io/projected/20f015bc-d781-4317-99c3-ce697c740441-kube-api-access-kpdmq\") pod \"20f015bc-d781-4317-99c3-ce697c740441\" (UID: \"20f015bc-d781-4317-99c3-ce697c740441\") "
Nov 04 03:41:50.430712 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:50.430549    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/20f015bc-d781-4317-99c3-ce697c740441-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20f015bc-d781-4317-99c3-ce697c740441-istio-podinfo\") pod \"20f015bc-d781-4317-99c3-ce697c740441\" (UID: \"20f015bc-d781-4317-99c3-ce697c740441\") "
Nov 04 03:41:50.430764 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:50.430718    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20f015bc-d781-4317-99c3-ce697c740441-credential-socket" pod "20f015bc-d781-4317-99c3-ce697c740441" (UID: "20f015bc-d781-4317-99c3-ce697c740441"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:41:50.431470 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:50.431068    5609 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20f015bc-d781-4317-99c3-ce697c740441-istiod-ca-cert\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:41:50.431470 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:50.431091    5609 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20f015bc-d781-4317-99c3-ce697c740441-credential-socket\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:41:50.431470 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:50.431105    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20f015bc-d781-4317-99c3-ce697c740441-workload-socket" pod "20f015bc-d781-4317-99c3-ce697c740441" (UID: "20f015bc-d781-4317-99c3-ce697c740441"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:41:50.431470 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:50.431326    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20f015bc-d781-4317-99c3-ce697c740441-istio-data" pod "20f015bc-d781-4317-99c3-ce697c740441" (UID: "20f015bc-d781-4317-99c3-ce697c740441"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:41:50.431470 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:50.431337    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20f015bc-d781-4317-99c3-ce697c740441-workload-certs" pod "20f015bc-d781-4317-99c3-ce697c740441" (UID: "20f015bc-d781-4317-99c3-ce697c740441"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:41:50.433863 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:50.433823    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/20f015bc-d781-4317-99c3-ce697c740441-istio-podinfo" pod "20f015bc-d781-4317-99c3-ce697c740441" (UID: "20f015bc-d781-4317-99c3-ce697c740441"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:41:50.433863 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:50.433846    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20f015bc-d781-4317-99c3-ce697c740441-istio-envoy" pod "20f015bc-d781-4317-99c3-ce697c740441" (UID: "20f015bc-d781-4317-99c3-ce697c740441"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:41:50.434041 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:50.433860    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20f015bc-d781-4317-99c3-ce697c740441-istio-token" pod "20f015bc-d781-4317-99c3-ce697c740441" (UID: "20f015bc-d781-4317-99c3-ce697c740441"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:41:50.434041 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:50.433865    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20f015bc-d781-4317-99c3-ce697c740441-kube-api-access-kpdmq" pod "20f015bc-d781-4317-99c3-ce697c740441" (UID: "20f015bc-d781-4317-99c3-ce697c740441"). InnerVolumeSpecName "kube-api-access-kpdmq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:41:50.532325 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:50.532295    5609 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20f015bc-d781-4317-99c3-ce697c740441-istio-envoy\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:41:50.532325 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:50.532324    5609 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20f015bc-d781-4317-99c3-ce697c740441-workload-socket\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:41:50.532546 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:50.532336    5609 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20f015bc-d781-4317-99c3-ce697c740441-istio-token\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:41:50.532546 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:50.532349    5609 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20f015bc-d781-4317-99c3-ce697c740441-istio-data\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:41:50.532546 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:50.532362    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kpdmq\" (UniqueName: \"kubernetes.io/projected/20f015bc-d781-4317-99c3-ce697c740441-kube-api-access-kpdmq\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:41:50.532546 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:50.532374    5609 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20f015bc-d781-4317-99c3-ce697c740441-istio-podinfo\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:41:50.532546 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:50.532385    5609 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20f015bc-d781-4317-99c3-ce697c740441-workload-certs\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:41:51.134915 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:51.134880    5609 generic.go:356] "Generic (PLEG): container finished" podID="20f015bc-d781-4317-99c3-ce697c740441" containerID="afab82e4b59537fe6181b47d2e50b98096f67715d10b0eeb13141eece72da517" exitCode=0
Nov 04 03:41:51.135304 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:51.134943    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v"
Nov 04 03:41:51.135304 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:51.134943    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v" event={"ID":"20f015bc-d781-4317-99c3-ce697c740441","Type":"ContainerDied","Data":"afab82e4b59537fe6181b47d2e50b98096f67715d10b0eeb13141eece72da517"}
Nov 04 03:41:51.135304 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:51.135012    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v" event={"ID":"20f015bc-d781-4317-99c3-ce697c740441","Type":"ContainerDied","Data":"9a480d6132b34046501f93fb90d6a572172aa33b41b326972571ef090ba02aeb"}
Nov 04 03:41:51.135304 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:51.135043    5609 scope.go:122] "RemoveContainer" containerID="afab82e4b59537fe6181b47d2e50b98096f67715d10b0eeb13141eece72da517"
Nov 04 03:41:51.149074 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:51.149050    5609 scope.go:122] "RemoveContainer" containerID="a41c54b503aa3d6a91a1fbaf959aa2a7f1d4aaba983e4222391160c984d4677d"
Nov 04 03:41:51.158828 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:51.158806    5609 scope.go:122] "RemoveContainer" containerID="afab82e4b59537fe6181b47d2e50b98096f67715d10b0eeb13141eece72da517"
Nov 04 03:41:51.159061 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:41:51.159045    5609 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afab82e4b59537fe6181b47d2e50b98096f67715d10b0eeb13141eece72da517\": container with ID starting with afab82e4b59537fe6181b47d2e50b98096f67715d10b0eeb13141eece72da517 not found: ID does not exist" containerID="afab82e4b59537fe6181b47d2e50b98096f67715d10b0eeb13141eece72da517"
Nov 04 03:41:51.159115 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:51.159069    5609 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afab82e4b59537fe6181b47d2e50b98096f67715d10b0eeb13141eece72da517"} err="failed to get container status \"afab82e4b59537fe6181b47d2e50b98096f67715d10b0eeb13141eece72da517\": rpc error: code = NotFound desc = could not find container \"afab82e4b59537fe6181b47d2e50b98096f67715d10b0eeb13141eece72da517\": container with ID starting with afab82e4b59537fe6181b47d2e50b98096f67715d10b0eeb13141eece72da517 not found: ID does not exist"
Nov 04 03:41:51.159115 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:51.159085    5609 scope.go:122] "RemoveContainer" containerID="a41c54b503aa3d6a91a1fbaf959aa2a7f1d4aaba983e4222391160c984d4677d"
Nov 04 03:41:51.159313 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:41:51.159293    5609 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a41c54b503aa3d6a91a1fbaf959aa2a7f1d4aaba983e4222391160c984d4677d\": container with ID starting with a41c54b503aa3d6a91a1fbaf959aa2a7f1d4aaba983e4222391160c984d4677d not found: ID does not exist" containerID="a41c54b503aa3d6a91a1fbaf959aa2a7f1d4aaba983e4222391160c984d4677d"
Nov 04 03:41:51.159371 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:51.159323    5609 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a41c54b503aa3d6a91a1fbaf959aa2a7f1d4aaba983e4222391160c984d4677d"} err="failed to get container status \"a41c54b503aa3d6a91a1fbaf959aa2a7f1d4aaba983e4222391160c984d4677d\": rpc error: code = NotFound desc = could not find container \"a41c54b503aa3d6a91a1fbaf959aa2a7f1d4aaba983e4222391160c984d4677d\": container with ID starting with a41c54b503aa3d6a91a1fbaf959aa2a7f1d4aaba983e4222391160c984d4677d not found: ID does not exist"
Nov 04 03:41:51.293284 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:51.293240    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v"]
Nov 04 03:41:51.297784 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:51.297752    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-8ecbc5-openshift-default-57b54b6745-lh58v"]
Nov 04 03:41:51.535381 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:51.535350    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="20f015bc-d781-4317-99c3-ce697c740441" path="/var/lib/kubelet/pods/20f015bc-d781-4317-99c3-ce697c740441/volumes"
Nov 04 03:41:51.884787 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:51.884682    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-1bf4bf-5bcc9466c5-b67d2"]
Nov 04 03:41:53.894806 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:53.894773    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-8ecbc5-776655854d-p7qp2"
Nov 04 03:41:53.988596 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:53.988572    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-8ecbc5-7df459b5c9-f2hdp"
Nov 04 03:41:53.991881 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:53.991856    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b263e963-52bb-4426-b6e6-e0b18f1ee810-kube-api-access-5dkp6\" (UniqueName: \"kubernetes.io/projected/b263e963-52bb-4426-b6e6-e0b18f1ee810-kube-api-access-5dkp6\") pod \"b263e963-52bb-4426-b6e6-e0b18f1ee810\" (UID: \"b263e963-52bb-4426-b6e6-e0b18f1ee810\") "
Nov 04 03:41:53.991993 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:53.991942    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f4096fc6-30ab-48db-82f5-85f9847473e0-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4096fc6-30ab-48db-82f5-85f9847473e0-plugins-config-volume\") pod \"f4096fc6-30ab-48db-82f5-85f9847473e0\" (UID: \"f4096fc6-30ab-48db-82f5-85f9847473e0\") "
Nov 04 03:41:53.992075 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:53.992017    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f4096fc6-30ab-48db-82f5-85f9847473e0-kube-api-access-mpjgz\" (UniqueName: \"kubernetes.io/projected/f4096fc6-30ab-48db-82f5-85f9847473e0-kube-api-access-mpjgz\") pod \"f4096fc6-30ab-48db-82f5-85f9847473e0\" (UID: \"f4096fc6-30ab-48db-82f5-85f9847473e0\") "
Nov 04 03:41:53.992131 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:53.992092    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b263e963-52bb-4426-b6e6-e0b18f1ee810-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b263e963-52bb-4426-b6e6-e0b18f1ee810-plugins-config-volume\") pod \"b263e963-52bb-4426-b6e6-e0b18f1ee810\" (UID: \"b263e963-52bb-4426-b6e6-e0b18f1ee810\") "
Nov 04 03:41:53.992327 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:53.992298    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4096fc6-30ab-48db-82f5-85f9847473e0-plugins-config-volume" pod "f4096fc6-30ab-48db-82f5-85f9847473e0" (UID: "f4096fc6-30ab-48db-82f5-85f9847473e0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:41:53.992496 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:53.992476    5609 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4096fc6-30ab-48db-82f5-85f9847473e0-plugins-config-volume\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:41:53.992738 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:53.992714    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b263e963-52bb-4426-b6e6-e0b18f1ee810-plugins-config-volume" pod "b263e963-52bb-4426-b6e6-e0b18f1ee810" (UID: "b263e963-52bb-4426-b6e6-e0b18f1ee810"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:41:53.994063 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:53.994035    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b263e963-52bb-4426-b6e6-e0b18f1ee810-kube-api-access-5dkp6" pod "b263e963-52bb-4426-b6e6-e0b18f1ee810" (UID: "b263e963-52bb-4426-b6e6-e0b18f1ee810"). InnerVolumeSpecName "kube-api-access-5dkp6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:41:53.994138 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:53.994093    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4096fc6-30ab-48db-82f5-85f9847473e0-kube-api-access-mpjgz" pod "f4096fc6-30ab-48db-82f5-85f9847473e0" (UID: "f4096fc6-30ab-48db-82f5-85f9847473e0"). InnerVolumeSpecName "kube-api-access-mpjgz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:41:54.094044 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:54.094007    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mpjgz\" (UniqueName: \"kubernetes.io/projected/f4096fc6-30ab-48db-82f5-85f9847473e0-kube-api-access-mpjgz\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:41:54.094044 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:54.094041    5609 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b263e963-52bb-4426-b6e6-e0b18f1ee810-plugins-config-volume\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:41:54.094255 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:54.094056    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5dkp6\" (UniqueName: \"kubernetes.io/projected/b263e963-52bb-4426-b6e6-e0b18f1ee810-kube-api-access-5dkp6\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:41:54.161333 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:54.161235    5609 generic.go:356] "Generic (PLEG): container finished" podID="f4096fc6-30ab-48db-82f5-85f9847473e0" containerID="1214221dd74bd9b042a302adf89e3cac1c8dcd8655770aea6f5e04515de85e17" exitCode=0
Nov 04 03:41:54.161333 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:54.161280    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-8ecbc5-776655854d-p7qp2"
Nov 04 03:41:54.161333 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:54.161311    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-8ecbc5-776655854d-p7qp2" event={"ID":"f4096fc6-30ab-48db-82f5-85f9847473e0","Type":"ContainerDied","Data":"1214221dd74bd9b042a302adf89e3cac1c8dcd8655770aea6f5e04515de85e17"}
Nov 04 03:41:54.161658 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:54.161371    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-8ecbc5-776655854d-p7qp2" event={"ID":"f4096fc6-30ab-48db-82f5-85f9847473e0","Type":"ContainerDied","Data":"44f3c80fbcf00f9177031ce07b6d3640452a491bf4d3b47ab42b40e4a47d2adb"}
Nov 04 03:41:54.161658 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:54.161421    5609 scope.go:122] "RemoveContainer" containerID="1214221dd74bd9b042a302adf89e3cac1c8dcd8655770aea6f5e04515de85e17"
Nov 04 03:41:54.163354 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:54.163330    5609 generic.go:356] "Generic (PLEG): container finished" podID="b263e963-52bb-4426-b6e6-e0b18f1ee810" containerID="8f914a65668790af43405e0226c3985efd8914101b4b46193e7ab513e6b79e26" exitCode=0
Nov 04 03:41:54.163567 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:54.163374    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-8ecbc5-7df459b5c9-f2hdp"
Nov 04 03:41:54.163671 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:54.163629    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-8ecbc5-7df459b5c9-f2hdp" event={"ID":"b263e963-52bb-4426-b6e6-e0b18f1ee810","Type":"ContainerDied","Data":"8f914a65668790af43405e0226c3985efd8914101b4b46193e7ab513e6b79e26"}
Nov 04 03:41:54.163738 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:54.163680    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-8ecbc5-7df459b5c9-f2hdp" event={"ID":"b263e963-52bb-4426-b6e6-e0b18f1ee810","Type":"ContainerDied","Data":"2c631e235303a912235f16d94d47c630b23936ef86b268b919de382b871ca267"}
Nov 04 03:41:54.175146 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:54.175005    5609 scope.go:122] "RemoveContainer" containerID="1214221dd74bd9b042a302adf89e3cac1c8dcd8655770aea6f5e04515de85e17"
Nov 04 03:41:54.175389 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:41:54.175361    5609 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1214221dd74bd9b042a302adf89e3cac1c8dcd8655770aea6f5e04515de85e17\": container with ID starting with 1214221dd74bd9b042a302adf89e3cac1c8dcd8655770aea6f5e04515de85e17 not found: ID does not exist" containerID="1214221dd74bd9b042a302adf89e3cac1c8dcd8655770aea6f5e04515de85e17"
Nov 04 03:41:54.175533 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:54.175429    5609 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1214221dd74bd9b042a302adf89e3cac1c8dcd8655770aea6f5e04515de85e17"} err="failed to get container status \"1214221dd74bd9b042a302adf89e3cac1c8dcd8655770aea6f5e04515de85e17\": rpc error: code = NotFound desc = could not find container \"1214221dd74bd9b042a302adf89e3cac1c8dcd8655770aea6f5e04515de85e17\": container with ID starting with 1214221dd74bd9b042a302adf89e3cac1c8dcd8655770aea6f5e04515de85e17 not found: ID does not exist"
Nov 04 03:41:54.175533 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:54.175454    5609 scope.go:122] "RemoveContainer" containerID="8f914a65668790af43405e0226c3985efd8914101b4b46193e7ab513e6b79e26"
Nov 04 03:41:54.184772 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:54.184756    5609 scope.go:122] "RemoveContainer" containerID="8f914a65668790af43405e0226c3985efd8914101b4b46193e7ab513e6b79e26"
Nov 04 03:41:54.185055 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:41:54.185033    5609 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f914a65668790af43405e0226c3985efd8914101b4b46193e7ab513e6b79e26\": container with ID starting with 8f914a65668790af43405e0226c3985efd8914101b4b46193e7ab513e6b79e26 not found: ID does not exist" containerID="8f914a65668790af43405e0226c3985efd8914101b4b46193e7ab513e6b79e26"
Nov 04 03:41:54.185173 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:54.185065    5609 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f914a65668790af43405e0226c3985efd8914101b4b46193e7ab513e6b79e26"} err="failed to get container status \"8f914a65668790af43405e0226c3985efd8914101b4b46193e7ab513e6b79e26\": rpc error: code = NotFound desc = could not find container \"8f914a65668790af43405e0226c3985efd8914101b4b46193e7ab513e6b79e26\": container with ID starting with 8f914a65668790af43405e0226c3985efd8914101b4b46193e7ab513e6b79e26 not found: ID does not exist"
Nov 04 03:41:54.319957 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:54.319921    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-8ecbc5-776655854d-p7qp2"]
Nov 04 03:41:54.324874 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:54.324845    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-8ecbc5-776655854d-p7qp2"]
Nov 04 03:41:54.425611 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:54.425532    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-8ecbc5-7df459b5c9-f2hdp"]
Nov 04 03:41:54.429898 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:54.429870    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-8ecbc5-7df459b5c9-f2hdp"]
Nov 04 03:41:55.595613 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:55.595577    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b263e963-52bb-4426-b6e6-e0b18f1ee810" path="/var/lib/kubelet/pods/b263e963-52bb-4426-b6e6-e0b18f1ee810/volumes"
Nov 04 03:41:55.596166 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:55.596147    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f4096fc6-30ab-48db-82f5-85f9847473e0" path="/var/lib/kubelet/pods/f4096fc6-30ab-48db-82f5-85f9847473e0/volumes"
Nov 04 03:41:56.888386 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:56.885438    5609 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-1bf4bf-5bcc9466c5-b67d2" podUID="b25bdf07-7285-440d-8cb5-1a34e9dea8cb" containerName="payload-pre-processing" containerID="cri-o://8607dd7ed0bc104d40135f56eeca8842c050178a3f76c2d07ca99050ffb66395" gracePeriod=25
Nov 04 03:41:57.150177 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:57.150119    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-1bf4bf-5bcc9466c5-b67d2"
Nov 04 03:41:57.199123 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:57.199089    5609 generic.go:356] "Generic (PLEG): container finished" podID="b25bdf07-7285-440d-8cb5-1a34e9dea8cb" containerID="8607dd7ed0bc104d40135f56eeca8842c050178a3f76c2d07ca99050ffb66395" exitCode=0
Nov 04 03:41:57.199292 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:57.199154    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-1bf4bf-5bcc9466c5-b67d2"
Nov 04 03:41:57.199292 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:57.199173    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-1bf4bf-5bcc9466c5-b67d2" event={"ID":"b25bdf07-7285-440d-8cb5-1a34e9dea8cb","Type":"ContainerDied","Data":"8607dd7ed0bc104d40135f56eeca8842c050178a3f76c2d07ca99050ffb66395"}
Nov 04 03:41:57.199292 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:57.199259    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-1bf4bf-5bcc9466c5-b67d2" event={"ID":"b25bdf07-7285-440d-8cb5-1a34e9dea8cb","Type":"ContainerDied","Data":"19d720d22b2bb76b68995f835a5532c5abbf16d145e217cf2b6d90e5eff80d73"}
Nov 04 03:41:57.199292 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:57.199284    5609 scope.go:122] "RemoveContainer" containerID="8607dd7ed0bc104d40135f56eeca8842c050178a3f76c2d07ca99050ffb66395"
Nov 04 03:41:57.209120 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:57.209103    5609 scope.go:122] "RemoveContainer" containerID="8607dd7ed0bc104d40135f56eeca8842c050178a3f76c2d07ca99050ffb66395"
Nov 04 03:41:57.209374 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:41:57.209354    5609 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8607dd7ed0bc104d40135f56eeca8842c050178a3f76c2d07ca99050ffb66395\": container with ID starting with 8607dd7ed0bc104d40135f56eeca8842c050178a3f76c2d07ca99050ffb66395 not found: ID does not exist" containerID="8607dd7ed0bc104d40135f56eeca8842c050178a3f76c2d07ca99050ffb66395"
Nov 04 03:41:57.209519 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:57.209381    5609 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8607dd7ed0bc104d40135f56eeca8842c050178a3f76c2d07ca99050ffb66395"} err="failed to get container status \"8607dd7ed0bc104d40135f56eeca8842c050178a3f76c2d07ca99050ffb66395\": rpc error: code = NotFound desc = could not find container \"8607dd7ed0bc104d40135f56eeca8842c050178a3f76c2d07ca99050ffb66395\": container with ID starting with 8607dd7ed0bc104d40135f56eeca8842c050178a3f76c2d07ca99050ffb66395 not found: ID does not exist"
Nov 04 03:41:57.234006 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:57.233985    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b25bdf07-7285-440d-8cb5-1a34e9dea8cb-kube-api-access-4kvlg\" (UniqueName: \"kubernetes.io/projected/b25bdf07-7285-440d-8cb5-1a34e9dea8cb-kube-api-access-4kvlg\") pod \"b25bdf07-7285-440d-8cb5-1a34e9dea8cb\" (UID: \"b25bdf07-7285-440d-8cb5-1a34e9dea8cb\") "
Nov 04 03:41:57.234121 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:57.234068    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b25bdf07-7285-440d-8cb5-1a34e9dea8cb-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b25bdf07-7285-440d-8cb5-1a34e9dea8cb-plugins-config-volume\") pod \"b25bdf07-7285-440d-8cb5-1a34e9dea8cb\" (UID: \"b25bdf07-7285-440d-8cb5-1a34e9dea8cb\") "
Nov 04 03:41:57.234479 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:57.234458    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b25bdf07-7285-440d-8cb5-1a34e9dea8cb-plugins-config-volume" pod "b25bdf07-7285-440d-8cb5-1a34e9dea8cb" (UID: "b25bdf07-7285-440d-8cb5-1a34e9dea8cb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:41:57.236065 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:57.236039    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b25bdf07-7285-440d-8cb5-1a34e9dea8cb-kube-api-access-4kvlg" pod "b25bdf07-7285-440d-8cb5-1a34e9dea8cb" (UID: "b25bdf07-7285-440d-8cb5-1a34e9dea8cb"). InnerVolumeSpecName "kube-api-access-4kvlg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:41:57.335588 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:57.335559    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4kvlg\" (UniqueName: \"kubernetes.io/projected/b25bdf07-7285-440d-8cb5-1a34e9dea8cb-kube-api-access-4kvlg\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:41:57.335588 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:57.335584    5609 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b25bdf07-7285-440d-8cb5-1a34e9dea8cb-plugins-config-volume\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:41:57.616460 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:57.616424    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-1bf4bf-5bcc9466c5-b67d2"]
Nov 04 03:41:57.624917 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:57.624881    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-1bf4bf-5bcc9466c5-b67d2"]
Nov 04 03:41:58.318668 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:58.318621    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-5d4032-6d7c8b496d-5rzsv"]
Nov 04 03:41:58.436049 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:58.434742    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-5d4032-854b9cd759-rcv54"]
Nov 04 03:41:59.536873 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:41:59.536834    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b25bdf07-7285-440d-8cb5-1a34e9dea8cb" path="/var/lib/kubelet/pods/b25bdf07-7285-440d-8cb5-1a34e9dea8cb/volumes"
Nov 04 03:42:03.319296 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:03.319236    5609 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-a-5d4032-6d7c8b496d-5rzsv" podUID="eda587fc-4491-448f-ae4a-1c8be7213bde" containerName="payload-processing" containerID="cri-o://156dd70e8bb480f1fb56ce107162945fa8b3fed805a5a7ae675af5cd8883c06d" gracePeriod=25
Nov 04 03:42:03.435191 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:03.435143    5609 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-5d4032-854b9cd759-rcv54" podUID="0f90496f-05cc-4b88-ab4f-a7d803235b1f" containerName="payload-pre-processing" containerID="cri-o://70e4b5b3f959ca3e0e5f5bf1557e89435b6c3fea0cf68b1043eaefab0c3e7c0e" gracePeriod=25
Nov 04 03:42:03.577086 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:03.577017    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-5d4032-6d7c8b496d-5rzsv"
Nov 04 03:42:03.622089 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:03.622058    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eda587fc-4491-448f-ae4a-1c8be7213bde-kube-api-access-s9qs6\" (UniqueName: \"kubernetes.io/projected/eda587fc-4491-448f-ae4a-1c8be7213bde-kube-api-access-s9qs6\") pod \"eda587fc-4491-448f-ae4a-1c8be7213bde\" (UID: \"eda587fc-4491-448f-ae4a-1c8be7213bde\") "
Nov 04 03:42:03.622222 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:03.622178    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eda587fc-4491-448f-ae4a-1c8be7213bde-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eda587fc-4491-448f-ae4a-1c8be7213bde-plugins-config-volume\") pod \"eda587fc-4491-448f-ae4a-1c8be7213bde\" (UID: \"eda587fc-4491-448f-ae4a-1c8be7213bde\") "
Nov 04 03:42:03.622655 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:03.622622    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eda587fc-4491-448f-ae4a-1c8be7213bde-plugins-config-volume" pod "eda587fc-4491-448f-ae4a-1c8be7213bde" (UID: "eda587fc-4491-448f-ae4a-1c8be7213bde"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:42:03.625434 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:03.625385    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eda587fc-4491-448f-ae4a-1c8be7213bde-kube-api-access-s9qs6" pod "eda587fc-4491-448f-ae4a-1c8be7213bde" (UID: "eda587fc-4491-448f-ae4a-1c8be7213bde"). InnerVolumeSpecName "kube-api-access-s9qs6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:42:03.679539 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:03.679513    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-5d4032-854b9cd759-rcv54"
Nov 04 03:42:03.723600 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:03.723563    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0f90496f-05cc-4b88-ab4f-a7d803235b1f-kube-api-access-vpl7t\" (UniqueName: \"kubernetes.io/projected/0f90496f-05cc-4b88-ab4f-a7d803235b1f-kube-api-access-vpl7t\") pod \"0f90496f-05cc-4b88-ab4f-a7d803235b1f\" (UID: \"0f90496f-05cc-4b88-ab4f-a7d803235b1f\") "
Nov 04 03:42:03.723754 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:03.723620    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0f90496f-05cc-4b88-ab4f-a7d803235b1f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f90496f-05cc-4b88-ab4f-a7d803235b1f-plugins-config-volume\") pod \"0f90496f-05cc-4b88-ab4f-a7d803235b1f\" (UID: \"0f90496f-05cc-4b88-ab4f-a7d803235b1f\") "
Nov 04 03:42:03.723988 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:03.723965    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f90496f-05cc-4b88-ab4f-a7d803235b1f-plugins-config-volume" pod "0f90496f-05cc-4b88-ab4f-a7d803235b1f" (UID: "0f90496f-05cc-4b88-ab4f-a7d803235b1f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:42:03.724263 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:03.724235    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s9qs6\" (UniqueName: \"kubernetes.io/projected/eda587fc-4491-448f-ae4a-1c8be7213bde-kube-api-access-s9qs6\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:03.724263 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:03.724260    5609 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eda587fc-4491-448f-ae4a-1c8be7213bde-plugins-config-volume\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:03.724440 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:03.724274    5609 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f90496f-05cc-4b88-ab4f-a7d803235b1f-plugins-config-volume\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:03.725828 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:03.725806    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f90496f-05cc-4b88-ab4f-a7d803235b1f-kube-api-access-vpl7t" pod "0f90496f-05cc-4b88-ab4f-a7d803235b1f" (UID: "0f90496f-05cc-4b88-ab4f-a7d803235b1f"). InnerVolumeSpecName "kube-api-access-vpl7t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:42:03.826171 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:03.826133    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vpl7t\" (UniqueName: \"kubernetes.io/projected/0f90496f-05cc-4b88-ab4f-a7d803235b1f-kube-api-access-vpl7t\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:04.258295 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:04.258263    5609 generic.go:356] "Generic (PLEG): container finished" podID="eda587fc-4491-448f-ae4a-1c8be7213bde" containerID="156dd70e8bb480f1fb56ce107162945fa8b3fed805a5a7ae675af5cd8883c06d" exitCode=0
Nov 04 03:42:04.258504 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:04.258308    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-5d4032-6d7c8b496d-5rzsv"
Nov 04 03:42:04.258504 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:04.258351    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-5d4032-6d7c8b496d-5rzsv" event={"ID":"eda587fc-4491-448f-ae4a-1c8be7213bde","Type":"ContainerDied","Data":"156dd70e8bb480f1fb56ce107162945fa8b3fed805a5a7ae675af5cd8883c06d"}
Nov 04 03:42:04.258504 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:04.258423    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-5d4032-6d7c8b496d-5rzsv" event={"ID":"eda587fc-4491-448f-ae4a-1c8be7213bde","Type":"ContainerDied","Data":"1c1bc79249c1ceefffb17c1ddc729510781129c128dab0203a509531fb0b73bc"}
Nov 04 03:42:04.258504 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:04.258427    5609 scope.go:122] "RemoveContainer" containerID="156dd70e8bb480f1fb56ce107162945fa8b3fed805a5a7ae675af5cd8883c06d"
Nov 04 03:42:04.259912 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:04.259887    5609 generic.go:356] "Generic (PLEG): container finished" podID="0f90496f-05cc-4b88-ab4f-a7d803235b1f" containerID="70e4b5b3f959ca3e0e5f5bf1557e89435b6c3fea0cf68b1043eaefab0c3e7c0e" exitCode=0
Nov 04 03:42:04.260038 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:04.259933    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-5d4032-854b9cd759-rcv54"
Nov 04 03:42:04.260038 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:04.259970    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-5d4032-854b9cd759-rcv54" event={"ID":"0f90496f-05cc-4b88-ab4f-a7d803235b1f","Type":"ContainerDied","Data":"70e4b5b3f959ca3e0e5f5bf1557e89435b6c3fea0cf68b1043eaefab0c3e7c0e"}
Nov 04 03:42:04.260038 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:04.260010    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-5d4032-854b9cd759-rcv54" event={"ID":"0f90496f-05cc-4b88-ab4f-a7d803235b1f","Type":"ContainerDied","Data":"6208d12929d65aaa6378665982c67bef3b0809a59613ec19d19b419316798430"}
Nov 04 03:42:04.269952 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:04.269932    5609 scope.go:122] "RemoveContainer" containerID="156dd70e8bb480f1fb56ce107162945fa8b3fed805a5a7ae675af5cd8883c06d"
Nov 04 03:42:04.270250 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:42:04.270233    5609 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"156dd70e8bb480f1fb56ce107162945fa8b3fed805a5a7ae675af5cd8883c06d\": container with ID starting with 156dd70e8bb480f1fb56ce107162945fa8b3fed805a5a7ae675af5cd8883c06d not found: ID does not exist" containerID="156dd70e8bb480f1fb56ce107162945fa8b3fed805a5a7ae675af5cd8883c06d"
Nov 04 03:42:04.270322 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:04.270260    5609 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"156dd70e8bb480f1fb56ce107162945fa8b3fed805a5a7ae675af5cd8883c06d"} err="failed to get container status \"156dd70e8bb480f1fb56ce107162945fa8b3fed805a5a7ae675af5cd8883c06d\": rpc error: code = NotFound desc = could not find container \"156dd70e8bb480f1fb56ce107162945fa8b3fed805a5a7ae675af5cd8883c06d\": container with ID starting with 156dd70e8bb480f1fb56ce107162945fa8b3fed805a5a7ae675af5cd8883c06d not found: ID does not exist"
Nov 04 03:42:04.270322 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:04.270278    5609 scope.go:122] "RemoveContainer" containerID="70e4b5b3f959ca3e0e5f5bf1557e89435b6c3fea0cf68b1043eaefab0c3e7c0e"
Nov 04 03:42:04.279582 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:04.279567    5609 scope.go:122] "RemoveContainer" containerID="70e4b5b3f959ca3e0e5f5bf1557e89435b6c3fea0cf68b1043eaefab0c3e7c0e"
Nov 04 03:42:04.279836 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:42:04.279817    5609 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70e4b5b3f959ca3e0e5f5bf1557e89435b6c3fea0cf68b1043eaefab0c3e7c0e\": container with ID starting with 70e4b5b3f959ca3e0e5f5bf1557e89435b6c3fea0cf68b1043eaefab0c3e7c0e not found: ID does not exist" containerID="70e4b5b3f959ca3e0e5f5bf1557e89435b6c3fea0cf68b1043eaefab0c3e7c0e"
Nov 04 03:42:04.279879 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:04.279843    5609 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70e4b5b3f959ca3e0e5f5bf1557e89435b6c3fea0cf68b1043eaefab0c3e7c0e"} err="failed to get container status \"70e4b5b3f959ca3e0e5f5bf1557e89435b6c3fea0cf68b1043eaefab0c3e7c0e\": rpc error: code = NotFound desc = could not find container \"70e4b5b3f959ca3e0e5f5bf1557e89435b6c3fea0cf68b1043eaefab0c3e7c0e\": container with ID starting with 70e4b5b3f959ca3e0e5f5bf1557e89435b6c3fea0cf68b1043eaefab0c3e7c0e not found: ID does not exist"
Nov 04 03:42:04.414704 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:04.414670    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-5d4032-6d7c8b496d-5rzsv"]
Nov 04 03:42:04.461804 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:04.461771    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-5d4032-6d7c8b496d-5rzsv"]
Nov 04 03:42:04.524313 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:04.524204    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-5d4032-854b9cd759-rcv54"]
Nov 04 03:42:04.528098 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:04.528072    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-5d4032-854b9cd759-rcv54"]
Nov 04 03:42:05.535498 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:05.535465    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0f90496f-05cc-4b88-ab4f-a7d803235b1f" path="/var/lib/kubelet/pods/0f90496f-05cc-4b88-ab4f-a7d803235b1f/volumes"
Nov 04 03:42:05.535964 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:05.535858    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eda587fc-4491-448f-ae4a-1c8be7213bde" path="/var/lib/kubelet/pods/eda587fc-4491-448f-ae4a-1c8be7213bde/volumes"
Nov 04 03:42:18.560348 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:42:18.560313    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:42:34.060960 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:34.060922    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-5d4032-7589d67fb-xml6f"]
Nov 04 03:42:34.061458 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:34.061135    5609 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5d4032-7589d67fb-xml6f" podUID="81dc8aa9-657a-4504-b306-177c32c1774e" containerName="maas-api" containerID="cri-o://e8dea0fcbb40175549a31928fd95702332def5c61ee9efa9741d64beb7bc250e" gracePeriod=30
Nov 04 03:42:34.404055 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:34.404030    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5d4032-7589d67fb-xml6f"
Nov 04 03:42:34.510085 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:34.510032    5609 generic.go:356] "Generic (PLEG): container finished" podID="81dc8aa9-657a-4504-b306-177c32c1774e" containerID="e8dea0fcbb40175549a31928fd95702332def5c61ee9efa9741d64beb7bc250e" exitCode=0
Nov 04 03:42:34.510284 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:34.510178    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5d4032-7589d67fb-xml6f" event={"ID":"81dc8aa9-657a-4504-b306-177c32c1774e","Type":"ContainerDied","Data":"e8dea0fcbb40175549a31928fd95702332def5c61ee9efa9741d64beb7bc250e"}
Nov 04 03:42:34.510284 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:34.510228    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5d4032-7589d67fb-xml6f" event={"ID":"81dc8aa9-657a-4504-b306-177c32c1774e","Type":"ContainerDied","Data":"e21cfe7779a1019271449db7a28deea889bf63a18a3f47ca7531b51058fc27e0"}
Nov 04 03:42:34.510284 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:34.510254    5609 scope.go:122] "RemoveContainer" containerID="e8dea0fcbb40175549a31928fd95702332def5c61ee9efa9741d64beb7bc250e"
Nov 04 03:42:34.510512 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:34.510454    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5d4032-7589d67fb-xml6f"
Nov 04 03:42:34.523303 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:34.523275    5609 scope.go:122] "RemoveContainer" containerID="e8dea0fcbb40175549a31928fd95702332def5c61ee9efa9741d64beb7bc250e"
Nov 04 03:42:34.523950 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:42:34.523922    5609 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8dea0fcbb40175549a31928fd95702332def5c61ee9efa9741d64beb7bc250e\": container with ID starting with e8dea0fcbb40175549a31928fd95702332def5c61ee9efa9741d64beb7bc250e not found: ID does not exist" containerID="e8dea0fcbb40175549a31928fd95702332def5c61ee9efa9741d64beb7bc250e"
Nov 04 03:42:34.524049 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:34.523959    5609 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8dea0fcbb40175549a31928fd95702332def5c61ee9efa9741d64beb7bc250e"} err="failed to get container status \"e8dea0fcbb40175549a31928fd95702332def5c61ee9efa9741d64beb7bc250e\": rpc error: code = NotFound desc = could not find container \"e8dea0fcbb40175549a31928fd95702332def5c61ee9efa9741d64beb7bc250e\": container with ID starting with e8dea0fcbb40175549a31928fd95702332def5c61ee9efa9741d64beb7bc250e not found: ID does not exist"
Nov 04 03:42:34.528386 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:34.528145    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/81dc8aa9-657a-4504-b306-177c32c1774e-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81dc8aa9-657a-4504-b306-177c32c1774e-openshift-service-ca\") pod \"81dc8aa9-657a-4504-b306-177c32c1774e\" (UID: \"81dc8aa9-657a-4504-b306-177c32c1774e\") "
Nov 04 03:42:34.528386 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:34.528281    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/81dc8aa9-657a-4504-b306-177c32c1774e-kube-api-access-bkrz6\" (UniqueName: \"kubernetes.io/projected/81dc8aa9-657a-4504-b306-177c32c1774e-kube-api-access-bkrz6\") pod \"81dc8aa9-657a-4504-b306-177c32c1774e\" (UID: \"81dc8aa9-657a-4504-b306-177c32c1774e\") "
Nov 04 03:42:34.528386 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:34.528319    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/81dc8aa9-657a-4504-b306-177c32c1774e-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81dc8aa9-657a-4504-b306-177c32c1774e-maas-api-tls\") pod \"81dc8aa9-657a-4504-b306-177c32c1774e\" (UID: \"81dc8aa9-657a-4504-b306-177c32c1774e\") "
Nov 04 03:42:34.528386 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:34.528353    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/81dc8aa9-657a-4504-b306-177c32c1774e-metrics-certs\" (UniqueName: \"kubernetes.io/secret/81dc8aa9-657a-4504-b306-177c32c1774e-metrics-certs\") pod \"81dc8aa9-657a-4504-b306-177c32c1774e\" (UID: \"81dc8aa9-657a-4504-b306-177c32c1774e\") "
Nov 04 03:42:34.529617 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:34.529248    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81dc8aa9-657a-4504-b306-177c32c1774e-openshift-service-ca" pod "81dc8aa9-657a-4504-b306-177c32c1774e" (UID: "81dc8aa9-657a-4504-b306-177c32c1774e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:42:34.533051 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:34.533020    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81dc8aa9-657a-4504-b306-177c32c1774e-metrics-certs" pod "81dc8aa9-657a-4504-b306-177c32c1774e" (UID: "81dc8aa9-657a-4504-b306-177c32c1774e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:42:34.533175 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:34.533064    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81dc8aa9-657a-4504-b306-177c32c1774e-kube-api-access-bkrz6" pod "81dc8aa9-657a-4504-b306-177c32c1774e" (UID: "81dc8aa9-657a-4504-b306-177c32c1774e"). InnerVolumeSpecName "kube-api-access-bkrz6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:42:34.533247 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:34.533110    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81dc8aa9-657a-4504-b306-177c32c1774e-maas-api-tls" pod "81dc8aa9-657a-4504-b306-177c32c1774e" (UID: "81dc8aa9-657a-4504-b306-177c32c1774e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:42:34.630670 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:34.630596    5609 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81dc8aa9-657a-4504-b306-177c32c1774e-openshift-service-ca\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:34.630670 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:34.630629    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bkrz6\" (UniqueName: \"kubernetes.io/projected/81dc8aa9-657a-4504-b306-177c32c1774e-kube-api-access-bkrz6\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:34.630670 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:34.630644    5609 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81dc8aa9-657a-4504-b306-177c32c1774e-maas-api-tls\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:34.630670 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:34.630660    5609 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81dc8aa9-657a-4504-b306-177c32c1774e-metrics-certs\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:34.927373 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:34.927342    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-5d4032-7589d67fb-xml6f"]
Nov 04 03:42:34.932654 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:34.932624    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-5d4032-7589d67fb-xml6f"]
Nov 04 03:42:35.233559 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:35.233477    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-1bf4bf-689dcc8854-wv6z4"]
Nov 04 03:42:35.234101 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:35.233664    5609 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-1bf4bf-689dcc8854-wv6z4" podUID="11207c06-566c-452c-a515-6d095242d4b5" containerName="maas-api" containerID="cri-o://b9313d1b7e27d519e5128c35615eb615b7cec6e01f5d2fffe4f1cdeceff28023" gracePeriod=30
Nov 04 03:42:35.552189 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:35.551977    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="81dc8aa9-657a-4504-b306-177c32c1774e" path="/var/lib/kubelet/pods/81dc8aa9-657a-4504-b306-177c32c1774e/volumes"
Nov 04 03:42:35.556128 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:35.555944    5609 generic.go:356] "Generic (PLEG): container finished" podID="11207c06-566c-452c-a515-6d095242d4b5" containerID="b9313d1b7e27d519e5128c35615eb615b7cec6e01f5d2fffe4f1cdeceff28023" exitCode=0
Nov 04 03:42:35.556128 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:35.556005    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-1bf4bf-689dcc8854-wv6z4" event={"ID":"11207c06-566c-452c-a515-6d095242d4b5","Type":"ContainerDied","Data":"b9313d1b7e27d519e5128c35615eb615b7cec6e01f5d2fffe4f1cdeceff28023"}
Nov 04 03:42:35.563491 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:35.563469    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-1bf4bf-689dcc8854-wv6z4"
Nov 04 03:42:35.646811 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:35.646781    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/11207c06-566c-452c-a515-6d095242d4b5-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11207c06-566c-452c-a515-6d095242d4b5-openshift-service-ca\") pod \"11207c06-566c-452c-a515-6d095242d4b5\" (UID: \"11207c06-566c-452c-a515-6d095242d4b5\") "
Nov 04 03:42:35.646973 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:35.646863    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/11207c06-566c-452c-a515-6d095242d4b5-metrics-certs\" (UniqueName: \"kubernetes.io/secret/11207c06-566c-452c-a515-6d095242d4b5-metrics-certs\") pod \"11207c06-566c-452c-a515-6d095242d4b5\" (UID: \"11207c06-566c-452c-a515-6d095242d4b5\") "
Nov 04 03:42:35.646973 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:35.646939    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/11207c06-566c-452c-a515-6d095242d4b5-kube-api-access-m66t8\" (UniqueName: \"kubernetes.io/projected/11207c06-566c-452c-a515-6d095242d4b5-kube-api-access-m66t8\") pod \"11207c06-566c-452c-a515-6d095242d4b5\" (UID: \"11207c06-566c-452c-a515-6d095242d4b5\") "
Nov 04 03:42:35.646973 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:35.646968    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/11207c06-566c-452c-a515-6d095242d4b5-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11207c06-566c-452c-a515-6d095242d4b5-maas-api-tls\") pod \"11207c06-566c-452c-a515-6d095242d4b5\" (UID: \"11207c06-566c-452c-a515-6d095242d4b5\") "
Nov 04 03:42:35.647190 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:35.647168    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11207c06-566c-452c-a515-6d095242d4b5-openshift-service-ca" pod "11207c06-566c-452c-a515-6d095242d4b5" (UID: "11207c06-566c-452c-a515-6d095242d4b5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:42:35.647301 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:35.647273    5609 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11207c06-566c-452c-a515-6d095242d4b5-openshift-service-ca\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:35.649128 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:35.649099    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11207c06-566c-452c-a515-6d095242d4b5-metrics-certs" pod "11207c06-566c-452c-a515-6d095242d4b5" (UID: "11207c06-566c-452c-a515-6d095242d4b5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:42:35.649128 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:35.649116    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11207c06-566c-452c-a515-6d095242d4b5-kube-api-access-m66t8" pod "11207c06-566c-452c-a515-6d095242d4b5" (UID: "11207c06-566c-452c-a515-6d095242d4b5"). InnerVolumeSpecName "kube-api-access-m66t8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:42:35.649265 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:35.649118    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11207c06-566c-452c-a515-6d095242d4b5-maas-api-tls" pod "11207c06-566c-452c-a515-6d095242d4b5" (UID: "11207c06-566c-452c-a515-6d095242d4b5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:42:35.748784 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:35.748750    5609 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11207c06-566c-452c-a515-6d095242d4b5-metrics-certs\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:35.748784 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:35.748782    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m66t8\" (UniqueName: \"kubernetes.io/projected/11207c06-566c-452c-a515-6d095242d4b5-kube-api-access-m66t8\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:35.748990 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:35.748796    5609 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11207c06-566c-452c-a515-6d095242d4b5-maas-api-tls\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:36.563435 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:36.563386    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-1bf4bf-689dcc8854-wv6z4"
Nov 04 03:42:36.563435 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:36.563423    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-1bf4bf-689dcc8854-wv6z4" event={"ID":"11207c06-566c-452c-a515-6d095242d4b5","Type":"ContainerDied","Data":"64985c82bb14c39b28c63f17cb8faeaef3c2d6f052e34a4ffeadb927240d99de"}
Nov 04 03:42:36.563925 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:36.563485    5609 scope.go:122] "RemoveContainer" containerID="b9313d1b7e27d519e5128c35615eb615b7cec6e01f5d2fffe4f1cdeceff28023"
Nov 04 03:42:36.720345 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:36.720311    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-1bf4bf-689dcc8854-wv6z4"]
Nov 04 03:42:36.743442 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:36.743412    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-1bf4bf-689dcc8854-wv6z4"]
Nov 04 03:42:37.535534 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:37.535505    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="11207c06-566c-452c-a515-6d095242d4b5" path="/var/lib/kubelet/pods/11207c06-566c-452c-a515-6d095242d4b5/volumes"
Nov 04 03:42:37.844842 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:37.844759    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-p9989_068619f8-7fbb-4b22-ac7d-a1f144e084c0/console-operator/2.log"
Nov 04 03:42:37.852977 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:37.852951    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vmw5z_21a42913-5c4e-4057-aae5-2804038f16e4/ovn-acl-logging/0.log"
Nov 04 03:42:37.869139 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:37.869112    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-p9989_068619f8-7fbb-4b22-ac7d-a1f144e084c0/console-operator/2.log"
Nov 04 03:42:37.883929 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:37.883906    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vmw5z_21a42913-5c4e-4057-aae5-2804038f16e4/ovn-acl-logging/0.log"
Nov 04 03:42:39.785593 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:39.785562    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk"]
Nov 04 03:42:39.785992 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:39.785773    5609 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk" podUID="1f315bbc-cddf-41ee-b044-92914a203e58" containerName="istio-proxy" containerID="cri-o://b6a924e6e695a68b9cc923df5fef390ae54d85acfaed4426fcaa8e280d9d7149" gracePeriod=30
Nov 04 03:42:41.174329 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:41.174301    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j"]
Nov 04 03:42:41.174756 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:41.174546    5609 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j" podUID="29087b39-1965-453c-9d19-61a4df9b0156" containerName="istio-proxy" containerID="cri-o://40e265ab2cc5138d454dbdecdf48920dea22d886f8f0056f51b5fc9acf1b0e63" gracePeriod=30
Nov 04 03:42:42.298692 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.298655    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-172a9554-sqjkm"]
Nov 04 03:42:42.299382 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.299363    5609 cpu_manager.go:414] "containerMap: removing container" podUID="20f015bc-d781-4317-99c3-ce697c740441" containerName="istio-proxy"
Nov 04 03:42:42.299466 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.299384    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="20f015bc-d781-4317-99c3-ce697c740441" containerName="istio-proxy"
Nov 04 03:42:42.299466 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.299413    5609 cpu_manager.go:414] "containerMap: removing container" podUID="ae56bf38-1c41-4c65-96c1-c723def99c82" containerName="payload-processing"
Nov 04 03:42:42.299466 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.299421    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ae56bf38-1c41-4c65-96c1-c723def99c82" containerName="payload-processing"
Nov 04 03:42:42.299466 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.299436    5609 cpu_manager.go:414] "containerMap: removing container" podUID="f4096fc6-30ab-48db-82f5-85f9847473e0" containerName="payload-processing"
Nov 04 03:42:42.299466 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.299445    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f4096fc6-30ab-48db-82f5-85f9847473e0" containerName="payload-processing"
Nov 04 03:42:42.299466 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.299456    5609 cpu_manager.go:414] "containerMap: removing container" podUID="1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99" containerName="istio-proxy"
Nov 04 03:42:42.299466 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.299461    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99" containerName="istio-proxy"
Nov 04 03:42:42.299682 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.299475    5609 cpu_manager.go:414] "containerMap: removing container" podUID="8003391e-2385-45df-b9cf-30d5c564a0c6" containerName="istio-proxy"
Nov 04 03:42:42.299682 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.299480    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8003391e-2385-45df-b9cf-30d5c564a0c6" containerName="istio-proxy"
Nov 04 03:42:42.299682 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.299487    5609 cpu_manager.go:414] "containerMap: removing container" podUID="eda587fc-4491-448f-ae4a-1c8be7213bde" containerName="payload-processing"
Nov 04 03:42:42.299682 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.299492    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eda587fc-4491-448f-ae4a-1c8be7213bde" containerName="payload-processing"
Nov 04 03:42:42.299682 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.299503    5609 cpu_manager.go:414] "containerMap: removing container" podUID="7dd4f492-1db4-4368-b0b3-23661c65b6d2" containerName="istio-proxy"
Nov 04 03:42:42.299682 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.299508    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7dd4f492-1db4-4368-b0b3-23661c65b6d2" containerName="istio-proxy"
Nov 04 03:42:42.299682 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.299520    5609 cpu_manager.go:414] "containerMap: removing container" podUID="297439ad-252f-487e-a47b-3cf22cddb1b6" containerName="istio-proxy"
Nov 04 03:42:42.299682 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.299524    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="297439ad-252f-487e-a47b-3cf22cddb1b6" containerName="istio-proxy"
Nov 04 03:42:42.299682 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.299532    5609 cpu_manager.go:414] "containerMap: removing container" podUID="d7610c80-027e-4494-8f93-2d275c06c796" containerName="maas-api"
Nov 04 03:42:42.299682 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.299537    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d7610c80-027e-4494-8f93-2d275c06c796" containerName="maas-api"
Nov 04 03:42:42.299682 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.299549    5609 cpu_manager.go:414] "containerMap: removing container" podUID="b263e963-52bb-4426-b6e6-e0b18f1ee810" containerName="payload-pre-processing"
Nov 04 03:42:42.299682 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.299553    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b263e963-52bb-4426-b6e6-e0b18f1ee810" containerName="payload-pre-processing"
Nov 04 03:42:42.299682 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.299560    5609 cpu_manager.go:414] "containerMap: removing container" podUID="b25bdf07-7285-440d-8cb5-1a34e9dea8cb" containerName="payload-pre-processing"
Nov 04 03:42:42.299682 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.299564    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b25bdf07-7285-440d-8cb5-1a34e9dea8cb" containerName="payload-pre-processing"
Nov 04 03:42:42.299682 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.299570    5609 cpu_manager.go:414] "containerMap: removing container" podUID="11207c06-566c-452c-a515-6d095242d4b5" containerName="maas-api"
Nov 04 03:42:42.299682 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.299574    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="11207c06-566c-452c-a515-6d095242d4b5" containerName="maas-api"
Nov 04 03:42:42.299682 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.299583    5609 cpu_manager.go:414] "containerMap: removing container" podUID="81dc8aa9-657a-4504-b306-177c32c1774e" containerName="maas-api"
Nov 04 03:42:42.299682 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.299587    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="81dc8aa9-657a-4504-b306-177c32c1774e" containerName="maas-api"
Nov 04 03:42:42.299682 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.299595    5609 cpu_manager.go:414] "containerMap: removing container" podUID="8003391e-2385-45df-b9cf-30d5c564a0c6" containerName="istio-proxy"
Nov 04 03:42:42.299682 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.299599    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8003391e-2385-45df-b9cf-30d5c564a0c6" containerName="istio-proxy"
Nov 04 03:42:42.299682 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.299605    5609 cpu_manager.go:414] "containerMap: removing container" podUID="0f90496f-05cc-4b88-ab4f-a7d803235b1f" containerName="payload-pre-processing"
Nov 04 03:42:42.299682 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.299610    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0f90496f-05cc-4b88-ab4f-a7d803235b1f" containerName="payload-pre-processing"
Nov 04 03:42:42.299682 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.299686    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="297439ad-252f-487e-a47b-3cf22cddb1b6" containerName="istio-proxy"
Nov 04 03:42:42.300328 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.299694    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="d7610c80-027e-4494-8f93-2d275c06c796" containerName="maas-api"
Nov 04 03:42:42.300328 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.299701    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="b263e963-52bb-4426-b6e6-e0b18f1ee810" containerName="payload-pre-processing"
Nov 04 03:42:42.300328 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.299709    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="b25bdf07-7285-440d-8cb5-1a34e9dea8cb" containerName="payload-pre-processing"
Nov 04 03:42:42.300328 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.299715    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="11207c06-566c-452c-a515-6d095242d4b5" containerName="maas-api"
Nov 04 03:42:42.300328 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.299720    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="20f015bc-d781-4317-99c3-ce697c740441" containerName="istio-proxy"
Nov 04 03:42:42.300328 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.299725    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="81dc8aa9-657a-4504-b306-177c32c1774e" containerName="maas-api"
Nov 04 03:42:42.300328 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.299731    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="8003391e-2385-45df-b9cf-30d5c564a0c6" containerName="istio-proxy"
Nov 04 03:42:42.300328 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.299735    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="0f90496f-05cc-4b88-ab4f-a7d803235b1f" containerName="payload-pre-processing"
Nov 04 03:42:42.300328 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.299742    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="ae56bf38-1c41-4c65-96c1-c723def99c82" containerName="payload-processing"
Nov 04 03:42:42.300328 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.299748    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="f4096fc6-30ab-48db-82f5-85f9847473e0" containerName="payload-processing"
Nov 04 03:42:42.300328 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.299754    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="1fa79428-b3e5-4d0f-a8f8-6f6d40e84e99" containerName="istio-proxy"
Nov 04 03:42:42.300328 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.299761    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="8003391e-2385-45df-b9cf-30d5c564a0c6" containerName="istio-proxy"
Nov 04 03:42:42.300328 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.299765    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="eda587fc-4491-448f-ae4a-1c8be7213bde" containerName="payload-processing"
Nov 04 03:42:42.300328 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.299771    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="7dd4f492-1db4-4368-b0b3-23661c65b6d2" containerName="istio-proxy"
Nov 04 03:42:42.304596 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.304574    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-172a9554-sqjkm"
Nov 04 03:42:42.401456 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.401412    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 03:42:42.401815 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.401637    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x9cnx\""
Nov 04 03:42:42.409153 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.409120    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-172a9554-sqjkm"]
Nov 04 03:42:42.481768 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.481155    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9320f6ca-389b-4857-86b3-5336f90445ce-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-172a9554-sqjkm\" (UID: \"9320f6ca-389b-4857-86b3-5336f90445ce\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-172a9554-sqjkm"
Nov 04 03:42:42.481768 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.481374    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9320f6ca-389b-4857-86b3-5336f90445ce-maas-api-xks-ca\") pod \"maas-api-revoke-sub-172a9554-sqjkm\" (UID: \"9320f6ca-389b-4857-86b3-5336f90445ce\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-172a9554-sqjkm"
Nov 04 03:42:42.481768 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.481446    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2ztd\" (UniqueName: \"kubernetes.io/projected/9320f6ca-389b-4857-86b3-5336f90445ce-kube-api-access-z2ztd\") pod \"maas-api-revoke-sub-172a9554-sqjkm\" (UID: \"9320f6ca-389b-4857-86b3-5336f90445ce\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-172a9554-sqjkm"
Nov 04 03:42:42.582684 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.582605    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9320f6ca-389b-4857-86b3-5336f90445ce-maas-api-xks-ca\") pod \"maas-api-revoke-sub-172a9554-sqjkm\" (UID: \"9320f6ca-389b-4857-86b3-5336f90445ce\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-172a9554-sqjkm"
Nov 04 03:42:42.582684 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.582654    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2ztd\" (UniqueName: \"kubernetes.io/projected/9320f6ca-389b-4857-86b3-5336f90445ce-kube-api-access-z2ztd\") pod \"maas-api-revoke-sub-172a9554-sqjkm\" (UID: \"9320f6ca-389b-4857-86b3-5336f90445ce\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-172a9554-sqjkm"
Nov 04 03:42:42.582881 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.582711    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9320f6ca-389b-4857-86b3-5336f90445ce-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-172a9554-sqjkm\" (UID: \"9320f6ca-389b-4857-86b3-5336f90445ce\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-172a9554-sqjkm"
Nov 04 03:42:42.583299 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.583278    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9320f6ca-389b-4857-86b3-5336f90445ce-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-172a9554-sqjkm\" (UID: \"9320f6ca-389b-4857-86b3-5336f90445ce\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-172a9554-sqjkm"
Nov 04 03:42:42.585047 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.585027    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9320f6ca-389b-4857-86b3-5336f90445ce-maas-api-xks-ca\") pod \"maas-api-revoke-sub-172a9554-sqjkm\" (UID: \"9320f6ca-389b-4857-86b3-5336f90445ce\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-172a9554-sqjkm"
Nov 04 03:42:42.640840 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.640810    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2ztd\" (UniqueName: \"kubernetes.io/projected/9320f6ca-389b-4857-86b3-5336f90445ce-kube-api-access-z2ztd\") pod \"maas-api-revoke-sub-172a9554-sqjkm\" (UID: \"9320f6ca-389b-4857-86b3-5336f90445ce\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-172a9554-sqjkm"
Nov 04 03:42:42.919505 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:42.919408    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-172a9554-sqjkm"
Nov 04 03:42:43.297490 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:43.297453    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-172a9554-sqjkm"]
Nov 04 03:42:43.301632 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:42:43.301597    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9320f6ca_389b_4857_86b3_5336f90445ce.slice/crio-9a27962180b6e3683064e87f4974b8c6490bd5eccf596366538fbf573a1f8e1f WatchSource:0}: Error finding container 9a27962180b6e3683064e87f4974b8c6490bd5eccf596366538fbf573a1f8e1f: Status 404 returned error can't find the container with id 9a27962180b6e3683064e87f4974b8c6490bd5eccf596366538fbf573a1f8e1f
Nov 04 03:42:43.619148 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:43.619117    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-172a9554-sqjkm" event={"ID":"9320f6ca-389b-4857-86b3-5336f90445ce","Type":"ContainerStarted","Data":"dd5e3127ca8be7a4618218a2b8256e80ca2fadd3ff92c9e39c4712cdc2024e3b"}
Nov 04 03:42:43.619280 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:43.619183    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-172a9554-sqjkm" event={"ID":"9320f6ca-389b-4857-86b3-5336f90445ce","Type":"ContainerStarted","Data":"9a27962180b6e3683064e87f4974b8c6490bd5eccf596366538fbf573a1f8e1f"}
Nov 04 03:42:43.729940 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:43.729890    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-172a9554-sqjkm" podStartSLOduration=1.729875483 podStartE2EDuration="1.729875483s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:42:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:42:43.728843042 +0000 UTC m=+1506.305495957" watchObservedRunningTime="2026-11-04 03:42:43.729875483 +0000 UTC m=+1506.306528430"
Nov 04 03:42:44.631612 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:44.631517    5609 generic.go:356] "Generic (PLEG): container finished" podID="9320f6ca-389b-4857-86b3-5336f90445ce" containerID="dd5e3127ca8be7a4618218a2b8256e80ca2fadd3ff92c9e39c4712cdc2024e3b" exitCode=0
Nov 04 03:42:44.632085 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:44.631605    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-172a9554-sqjkm" event={"ID":"9320f6ca-389b-4857-86b3-5336f90445ce","Type":"ContainerDied","Data":"dd5e3127ca8be7a4618218a2b8256e80ca2fadd3ff92c9e39c4712cdc2024e3b"}
Nov 04 03:42:45.074122 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.074096    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk"
Nov 04 03:42:45.116745 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.116705    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f315bbc-cddf-41ee-b044-92914a203e58-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f315bbc-cddf-41ee-b044-92914a203e58-workload-socket\") pod \"1f315bbc-cddf-41ee-b044-92914a203e58\" (UID: \"1f315bbc-cddf-41ee-b044-92914a203e58\") "
Nov 04 03:42:45.116918 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.116766    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f315bbc-cddf-41ee-b044-92914a203e58-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f315bbc-cddf-41ee-b044-92914a203e58-workload-certs\") pod \"1f315bbc-cddf-41ee-b044-92914a203e58\" (UID: \"1f315bbc-cddf-41ee-b044-92914a203e58\") "
Nov 04 03:42:45.116918 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.116803    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f315bbc-cddf-41ee-b044-92914a203e58-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f315bbc-cddf-41ee-b044-92914a203e58-istio-envoy\") pod \"1f315bbc-cddf-41ee-b044-92914a203e58\" (UID: \"1f315bbc-cddf-41ee-b044-92914a203e58\") "
Nov 04 03:42:45.116918 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.116861    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1f315bbc-cddf-41ee-b044-92914a203e58-kube-api-access-lxlwk\" (UniqueName: \"kubernetes.io/projected/1f315bbc-cddf-41ee-b044-92914a203e58-kube-api-access-lxlwk\") pod \"1f315bbc-cddf-41ee-b044-92914a203e58\" (UID: \"1f315bbc-cddf-41ee-b044-92914a203e58\") "
Nov 04 03:42:45.116918 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.116917    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f315bbc-cddf-41ee-b044-92914a203e58-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f315bbc-cddf-41ee-b044-92914a203e58-istio-data\") pod \"1f315bbc-cddf-41ee-b044-92914a203e58\" (UID: \"1f315bbc-cddf-41ee-b044-92914a203e58\") "
Nov 04 03:42:45.117121 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.116982    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f315bbc-cddf-41ee-b044-92914a203e58-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f315bbc-cddf-41ee-b044-92914a203e58-credential-socket\") pod \"1f315bbc-cddf-41ee-b044-92914a203e58\" (UID: \"1f315bbc-cddf-41ee-b044-92914a203e58\") "
Nov 04 03:42:45.117121 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.116985    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f315bbc-cddf-41ee-b044-92914a203e58-workload-socket" pod "1f315bbc-cddf-41ee-b044-92914a203e58" (UID: "1f315bbc-cddf-41ee-b044-92914a203e58"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:42:45.117121 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.117028    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1f315bbc-cddf-41ee-b044-92914a203e58-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f315bbc-cddf-41ee-b044-92914a203e58-istio-podinfo\") pod \"1f315bbc-cddf-41ee-b044-92914a203e58\" (UID: \"1f315bbc-cddf-41ee-b044-92914a203e58\") "
Nov 04 03:42:45.117121 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.117058    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1f315bbc-cddf-41ee-b044-92914a203e58-istio-token\" (UniqueName: \"kubernetes.io/projected/1f315bbc-cddf-41ee-b044-92914a203e58-istio-token\") pod \"1f315bbc-cddf-41ee-b044-92914a203e58\" (UID: \"1f315bbc-cddf-41ee-b044-92914a203e58\") "
Nov 04 03:42:45.117121 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.117098    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f315bbc-cddf-41ee-b044-92914a203e58-workload-certs" pod "1f315bbc-cddf-41ee-b044-92914a203e58" (UID: "1f315bbc-cddf-41ee-b044-92914a203e58"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:42:45.117365 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.117128    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1f315bbc-cddf-41ee-b044-92914a203e58-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f315bbc-cddf-41ee-b044-92914a203e58-istiod-ca-cert\") pod \"1f315bbc-cddf-41ee-b044-92914a203e58\" (UID: \"1f315bbc-cddf-41ee-b044-92914a203e58\") "
Nov 04 03:42:45.117365 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.117200    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f315bbc-cddf-41ee-b044-92914a203e58-istio-data" pod "1f315bbc-cddf-41ee-b044-92914a203e58" (UID: "1f315bbc-cddf-41ee-b044-92914a203e58"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:42:45.117505 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.117377    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f315bbc-cddf-41ee-b044-92914a203e58-credential-socket" pod "1f315bbc-cddf-41ee-b044-92914a203e58" (UID: "1f315bbc-cddf-41ee-b044-92914a203e58"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:42:45.117587 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.117568    5609 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f315bbc-cddf-41ee-b044-92914a203e58-workload-socket\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:45.117647 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.117591    5609 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f315bbc-cddf-41ee-b044-92914a203e58-workload-certs\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:45.117647 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.117595    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f315bbc-cddf-41ee-b044-92914a203e58-istiod-ca-cert" pod "1f315bbc-cddf-41ee-b044-92914a203e58" (UID: "1f315bbc-cddf-41ee-b044-92914a203e58"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:42:45.117647 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.117605    5609 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f315bbc-cddf-41ee-b044-92914a203e58-istio-data\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:45.117647 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.117619    5609 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f315bbc-cddf-41ee-b044-92914a203e58-credential-socket\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:45.119521 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.119493    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1f315bbc-cddf-41ee-b044-92914a203e58-istio-podinfo" pod "1f315bbc-cddf-41ee-b044-92914a203e58" (UID: "1f315bbc-cddf-41ee-b044-92914a203e58"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:42:45.119764 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.119738    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f315bbc-cddf-41ee-b044-92914a203e58-istio-token" pod "1f315bbc-cddf-41ee-b044-92914a203e58" (UID: "1f315bbc-cddf-41ee-b044-92914a203e58"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:42:45.119939 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.119924    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f315bbc-cddf-41ee-b044-92914a203e58-istio-envoy" pod "1f315bbc-cddf-41ee-b044-92914a203e58" (UID: "1f315bbc-cddf-41ee-b044-92914a203e58"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:42:45.120038 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.120023    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f315bbc-cddf-41ee-b044-92914a203e58-kube-api-access-lxlwk" pod "1f315bbc-cddf-41ee-b044-92914a203e58" (UID: "1f315bbc-cddf-41ee-b044-92914a203e58"). InnerVolumeSpecName "kube-api-access-lxlwk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:42:45.218513 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.218479    5609 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f315bbc-cddf-41ee-b044-92914a203e58-istio-envoy\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:45.218513 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.218515    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxlwk\" (UniqueName: \"kubernetes.io/projected/1f315bbc-cddf-41ee-b044-92914a203e58-kube-api-access-lxlwk\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:45.218770 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.218531    5609 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f315bbc-cddf-41ee-b044-92914a203e58-istio-podinfo\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:45.218770 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.218545    5609 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f315bbc-cddf-41ee-b044-92914a203e58-istio-token\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:45.218770 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.218559    5609 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f315bbc-cddf-41ee-b044-92914a203e58-istiod-ca-cert\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:45.642678 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.642647    5609 generic.go:356] "Generic (PLEG): container finished" podID="1f315bbc-cddf-41ee-b044-92914a203e58" containerID="b6a924e6e695a68b9cc923df5fef390ae54d85acfaed4426fcaa8e280d9d7149" exitCode=0
Nov 04 03:42:45.643119 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.642719    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk"
Nov 04 03:42:45.643119 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.642733    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk" event={"ID":"1f315bbc-cddf-41ee-b044-92914a203e58","Type":"ContainerDied","Data":"b6a924e6e695a68b9cc923df5fef390ae54d85acfaed4426fcaa8e280d9d7149"}
Nov 04 03:42:45.643119 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.642860    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk" event={"ID":"1f315bbc-cddf-41ee-b044-92914a203e58","Type":"ContainerDied","Data":"b677db0951b336fb9e9ef1abc6a38163829e3a6752f0db5ccc0eb6b1e0a1c2f1"}
Nov 04 03:42:45.643119 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.642886    5609 scope.go:122] "RemoveContainer" containerID="b6a924e6e695a68b9cc923df5fef390ae54d85acfaed4426fcaa8e280d9d7149"
Nov 04 03:42:45.657934 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.657911    5609 scope.go:122] "RemoveContainer" containerID="cade2663d5901a6475aae1eb80d2a270fcca95a241dd92b627656ff6db176ea7"
Nov 04 03:42:45.673560 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.672675    5609 scope.go:122] "RemoveContainer" containerID="b6a924e6e695a68b9cc923df5fef390ae54d85acfaed4426fcaa8e280d9d7149"
Nov 04 03:42:45.673560 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:42:45.672989    5609 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6a924e6e695a68b9cc923df5fef390ae54d85acfaed4426fcaa8e280d9d7149\": container with ID starting with b6a924e6e695a68b9cc923df5fef390ae54d85acfaed4426fcaa8e280d9d7149 not found: ID does not exist" containerID="b6a924e6e695a68b9cc923df5fef390ae54d85acfaed4426fcaa8e280d9d7149"
Nov 04 03:42:45.673560 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.673017    5609 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6a924e6e695a68b9cc923df5fef390ae54d85acfaed4426fcaa8e280d9d7149"} err="failed to get container status \"b6a924e6e695a68b9cc923df5fef390ae54d85acfaed4426fcaa8e280d9d7149\": rpc error: code = NotFound desc = could not find container \"b6a924e6e695a68b9cc923df5fef390ae54d85acfaed4426fcaa8e280d9d7149\": container with ID starting with b6a924e6e695a68b9cc923df5fef390ae54d85acfaed4426fcaa8e280d9d7149 not found: ID does not exist"
Nov 04 03:42:45.673560 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.673034    5609 scope.go:122] "RemoveContainer" containerID="cade2663d5901a6475aae1eb80d2a270fcca95a241dd92b627656ff6db176ea7"
Nov 04 03:42:45.673560 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:42:45.673282    5609 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cade2663d5901a6475aae1eb80d2a270fcca95a241dd92b627656ff6db176ea7\": container with ID starting with cade2663d5901a6475aae1eb80d2a270fcca95a241dd92b627656ff6db176ea7 not found: ID does not exist" containerID="cade2663d5901a6475aae1eb80d2a270fcca95a241dd92b627656ff6db176ea7"
Nov 04 03:42:45.673560 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.673313    5609 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cade2663d5901a6475aae1eb80d2a270fcca95a241dd92b627656ff6db176ea7"} err="failed to get container status \"cade2663d5901a6475aae1eb80d2a270fcca95a241dd92b627656ff6db176ea7\": rpc error: code = NotFound desc = could not find container \"cade2663d5901a6475aae1eb80d2a270fcca95a241dd92b627656ff6db176ea7\": container with ID starting with cade2663d5901a6475aae1eb80d2a270fcca95a241dd92b627656ff6db176ea7 not found: ID does not exist"
Nov 04 03:42:45.811844 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.807759    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-172a9554-sqjkm"
Nov 04 03:42:45.811844 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.811699    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk"]
Nov 04 03:42:45.839878 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.839839    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9320f6ca-389b-4857-86b3-5336f90445ce-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9320f6ca-389b-4857-86b3-5336f90445ce-maas-api-openshift-ca\") pod \"9320f6ca-389b-4857-86b3-5336f90445ce\" (UID: \"9320f6ca-389b-4857-86b3-5336f90445ce\") "
Nov 04 03:42:45.840047 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.839914    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9320f6ca-389b-4857-86b3-5336f90445ce-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9320f6ca-389b-4857-86b3-5336f90445ce-maas-api-xks-ca\") pod \"9320f6ca-389b-4857-86b3-5336f90445ce\" (UID: \"9320f6ca-389b-4857-86b3-5336f90445ce\") "
Nov 04 03:42:45.840047 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.840010    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9320f6ca-389b-4857-86b3-5336f90445ce-kube-api-access-z2ztd\" (UniqueName: \"kubernetes.io/projected/9320f6ca-389b-4857-86b3-5336f90445ce-kube-api-access-z2ztd\") pod \"9320f6ca-389b-4857-86b3-5336f90445ce\" (UID: \"9320f6ca-389b-4857-86b3-5336f90445ce\") "
Nov 04 03:42:45.840264 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.840239    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9320f6ca-389b-4857-86b3-5336f90445ce-maas-api-openshift-ca" pod "9320f6ca-389b-4857-86b3-5336f90445ce" (UID: "9320f6ca-389b-4857-86b3-5336f90445ce"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:42:45.840528 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.840510    5609 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9320f6ca-389b-4857-86b3-5336f90445ce-maas-api-openshift-ca\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:45.842323 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.842297    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9320f6ca-389b-4857-86b3-5336f90445ce-maas-api-xks-ca" pod "9320f6ca-389b-4857-86b3-5336f90445ce" (UID: "9320f6ca-389b-4857-86b3-5336f90445ce"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:42:45.842491 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.842466    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9320f6ca-389b-4857-86b3-5336f90445ce-kube-api-access-z2ztd" pod "9320f6ca-389b-4857-86b3-5336f90445ce" (UID: "9320f6ca-389b-4857-86b3-5336f90445ce"). InnerVolumeSpecName "kube-api-access-z2ztd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:42:45.854144 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.854101    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-5d4032-openshift-default-5c878f7d45-67qnk"]
Nov 04 03:42:45.942205 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.942157    5609 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9320f6ca-389b-4857-86b3-5336f90445ce-maas-api-xks-ca\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:45.942205 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:45.942194    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z2ztd\" (UniqueName: \"kubernetes.io/projected/9320f6ca-389b-4857-86b3-5336f90445ce-kube-api-access-z2ztd\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:46.455481 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:46.455413    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j"
Nov 04 03:42:46.558424 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:46.557758    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/29087b39-1965-453c-9d19-61a4df9b0156-kube-api-access-45m98\" (UniqueName: \"kubernetes.io/projected/29087b39-1965-453c-9d19-61a4df9b0156-kube-api-access-45m98\") pod \"29087b39-1965-453c-9d19-61a4df9b0156\" (UID: \"29087b39-1965-453c-9d19-61a4df9b0156\") "
Nov 04 03:42:46.558424 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:46.557844    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29087b39-1965-453c-9d19-61a4df9b0156-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29087b39-1965-453c-9d19-61a4df9b0156-credential-socket\") pod \"29087b39-1965-453c-9d19-61a4df9b0156\" (UID: \"29087b39-1965-453c-9d19-61a4df9b0156\") "
Nov 04 03:42:46.558424 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:46.557932    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29087b39-1965-453c-9d19-61a4df9b0156-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29087b39-1965-453c-9d19-61a4df9b0156-istio-data\") pod \"29087b39-1965-453c-9d19-61a4df9b0156\" (UID: \"29087b39-1965-453c-9d19-61a4df9b0156\") "
Nov 04 03:42:46.558424 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:46.557967    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29087b39-1965-453c-9d19-61a4df9b0156-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29087b39-1965-453c-9d19-61a4df9b0156-workload-socket\") pod \"29087b39-1965-453c-9d19-61a4df9b0156\" (UID: \"29087b39-1965-453c-9d19-61a4df9b0156\") "
Nov 04 03:42:46.558424 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:46.558049    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29087b39-1965-453c-9d19-61a4df9b0156-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29087b39-1965-453c-9d19-61a4df9b0156-istio-envoy\") pod \"29087b39-1965-453c-9d19-61a4df9b0156\" (UID: \"29087b39-1965-453c-9d19-61a4df9b0156\") "
Nov 04 03:42:46.558424 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:46.558087    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29087b39-1965-453c-9d19-61a4df9b0156-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29087b39-1965-453c-9d19-61a4df9b0156-workload-certs\") pod \"29087b39-1965-453c-9d19-61a4df9b0156\" (UID: \"29087b39-1965-453c-9d19-61a4df9b0156\") "
Nov 04 03:42:46.558424 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:46.558116    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/29087b39-1965-453c-9d19-61a4df9b0156-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29087b39-1965-453c-9d19-61a4df9b0156-istiod-ca-cert\") pod \"29087b39-1965-453c-9d19-61a4df9b0156\" (UID: \"29087b39-1965-453c-9d19-61a4df9b0156\") "
Nov 04 03:42:46.558424 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:46.558199    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/29087b39-1965-453c-9d19-61a4df9b0156-istio-token\" (UniqueName: \"kubernetes.io/projected/29087b39-1965-453c-9d19-61a4df9b0156-istio-token\") pod \"29087b39-1965-453c-9d19-61a4df9b0156\" (UID: \"29087b39-1965-453c-9d19-61a4df9b0156\") "
Nov 04 03:42:46.558424 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:46.558251    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/29087b39-1965-453c-9d19-61a4df9b0156-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29087b39-1965-453c-9d19-61a4df9b0156-istio-podinfo\") pod \"29087b39-1965-453c-9d19-61a4df9b0156\" (UID: \"29087b39-1965-453c-9d19-61a4df9b0156\") "
Nov 04 03:42:46.589016 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:46.588980    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29087b39-1965-453c-9d19-61a4df9b0156-credential-socket" pod "29087b39-1965-453c-9d19-61a4df9b0156" (UID: "29087b39-1965-453c-9d19-61a4df9b0156"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:42:46.589442 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:46.589419    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29087b39-1965-453c-9d19-61a4df9b0156-istio-data" pod "29087b39-1965-453c-9d19-61a4df9b0156" (UID: "29087b39-1965-453c-9d19-61a4df9b0156"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:42:46.594435 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:46.592626    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29087b39-1965-453c-9d19-61a4df9b0156-istiod-ca-cert" pod "29087b39-1965-453c-9d19-61a4df9b0156" (UID: "29087b39-1965-453c-9d19-61a4df9b0156"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:42:46.594435 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:46.592701    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29087b39-1965-453c-9d19-61a4df9b0156-workload-certs" pod "29087b39-1965-453c-9d19-61a4df9b0156" (UID: "29087b39-1965-453c-9d19-61a4df9b0156"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:42:46.611424 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:46.606125    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/29087b39-1965-453c-9d19-61a4df9b0156-istio-podinfo" pod "29087b39-1965-453c-9d19-61a4df9b0156" (UID: "29087b39-1965-453c-9d19-61a4df9b0156"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:42:46.611424 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:46.606379    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29087b39-1965-453c-9d19-61a4df9b0156-workload-socket" pod "29087b39-1965-453c-9d19-61a4df9b0156" (UID: "29087b39-1965-453c-9d19-61a4df9b0156"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:42:46.611424 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:46.607595    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29087b39-1965-453c-9d19-61a4df9b0156-istio-token" pod "29087b39-1965-453c-9d19-61a4df9b0156" (UID: "29087b39-1965-453c-9d19-61a4df9b0156"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:42:46.611424 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:46.607717    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29087b39-1965-453c-9d19-61a4df9b0156-istio-envoy" pod "29087b39-1965-453c-9d19-61a4df9b0156" (UID: "29087b39-1965-453c-9d19-61a4df9b0156"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:42:46.611424 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:46.609557    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29087b39-1965-453c-9d19-61a4df9b0156-kube-api-access-45m98" pod "29087b39-1965-453c-9d19-61a4df9b0156" (UID: "29087b39-1965-453c-9d19-61a4df9b0156"). InnerVolumeSpecName "kube-api-access-45m98". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:42:46.652564 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:46.652524    5609 generic.go:356] "Generic (PLEG): container finished" podID="29087b39-1965-453c-9d19-61a4df9b0156" containerID="40e265ab2cc5138d454dbdecdf48920dea22d886f8f0056f51b5fc9acf1b0e63" exitCode=0
Nov 04 03:42:46.652947 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:46.652639    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j" event={"ID":"29087b39-1965-453c-9d19-61a4df9b0156","Type":"ContainerDied","Data":"40e265ab2cc5138d454dbdecdf48920dea22d886f8f0056f51b5fc9acf1b0e63"}
Nov 04 03:42:46.652947 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:46.652678    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j" event={"ID":"29087b39-1965-453c-9d19-61a4df9b0156","Type":"ContainerDied","Data":"c944108eff2324d03659ecb1a240bfd0ea6ddd398b4e1ed6ab3bc445738d6765"}
Nov 04 03:42:46.652947 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:46.652699    5609 scope.go:122] "RemoveContainer" containerID="40e265ab2cc5138d454dbdecdf48920dea22d886f8f0056f51b5fc9acf1b0e63"
Nov 04 03:42:46.652947 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:46.652789    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j"
Nov 04 03:42:46.659327 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:46.659306    5609 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29087b39-1965-453c-9d19-61a4df9b0156-istio-token\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:46.659327 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:46.659327    5609 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29087b39-1965-453c-9d19-61a4df9b0156-istio-podinfo\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:46.659534 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:46.659336    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-45m98\" (UniqueName: \"kubernetes.io/projected/29087b39-1965-453c-9d19-61a4df9b0156-kube-api-access-45m98\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:46.659534 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:46.659346    5609 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29087b39-1965-453c-9d19-61a4df9b0156-credential-socket\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:46.659534 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:46.659354    5609 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29087b39-1965-453c-9d19-61a4df9b0156-istio-data\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:46.659534 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:46.659368    5609 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29087b39-1965-453c-9d19-61a4df9b0156-workload-socket\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:46.659534 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:46.659377    5609 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29087b39-1965-453c-9d19-61a4df9b0156-istio-envoy\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:46.659534 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:46.659385    5609 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29087b39-1965-453c-9d19-61a4df9b0156-workload-certs\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:46.659534 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:46.659392    5609 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29087b39-1965-453c-9d19-61a4df9b0156-istiod-ca-cert\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:46.660684 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:46.660149    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-172a9554-sqjkm" event={"ID":"9320f6ca-389b-4857-86b3-5336f90445ce","Type":"ContainerDied","Data":"9a27962180b6e3683064e87f4974b8c6490bd5eccf596366538fbf573a1f8e1f"}
Nov 04 03:42:46.660684 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:46.660184    5609 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9a27962180b6e3683064e87f4974b8c6490bd5eccf596366538fbf573a1f8e1f"
Nov 04 03:42:46.660684 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:46.660232    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-172a9554-sqjkm"
Nov 04 03:42:46.681136 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:46.680502    5609 scope.go:122] "RemoveContainer" containerID="5b5e94fd5bd8f91a89507c85487eae802f5a03c0034a9904018094c342b31060"
Nov 04 03:42:46.691859 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:46.691837    5609 scope.go:122] "RemoveContainer" containerID="40e265ab2cc5138d454dbdecdf48920dea22d886f8f0056f51b5fc9acf1b0e63"
Nov 04 03:42:46.692177 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:42:46.692151    5609 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40e265ab2cc5138d454dbdecdf48920dea22d886f8f0056f51b5fc9acf1b0e63\": container with ID starting with 40e265ab2cc5138d454dbdecdf48920dea22d886f8f0056f51b5fc9acf1b0e63 not found: ID does not exist" containerID="40e265ab2cc5138d454dbdecdf48920dea22d886f8f0056f51b5fc9acf1b0e63"
Nov 04 03:42:46.692277 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:46.692180    5609 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40e265ab2cc5138d454dbdecdf48920dea22d886f8f0056f51b5fc9acf1b0e63"} err="failed to get container status \"40e265ab2cc5138d454dbdecdf48920dea22d886f8f0056f51b5fc9acf1b0e63\": rpc error: code = NotFound desc = could not find container \"40e265ab2cc5138d454dbdecdf48920dea22d886f8f0056f51b5fc9acf1b0e63\": container with ID starting with 40e265ab2cc5138d454dbdecdf48920dea22d886f8f0056f51b5fc9acf1b0e63 not found: ID does not exist"
Nov 04 03:42:46.692277 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:46.692199    5609 scope.go:122] "RemoveContainer" containerID="5b5e94fd5bd8f91a89507c85487eae802f5a03c0034a9904018094c342b31060"
Nov 04 03:42:46.692553 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:42:46.692444    5609 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b5e94fd5bd8f91a89507c85487eae802f5a03c0034a9904018094c342b31060\": container with ID starting with 5b5e94fd5bd8f91a89507c85487eae802f5a03c0034a9904018094c342b31060 not found: ID does not exist" containerID="5b5e94fd5bd8f91a89507c85487eae802f5a03c0034a9904018094c342b31060"
Nov 04 03:42:46.692553 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:46.692465    5609 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b5e94fd5bd8f91a89507c85487eae802f5a03c0034a9904018094c342b31060"} err="failed to get container status \"5b5e94fd5bd8f91a89507c85487eae802f5a03c0034a9904018094c342b31060\": rpc error: code = NotFound desc = could not find container \"5b5e94fd5bd8f91a89507c85487eae802f5a03c0034a9904018094c342b31060\": container with ID starting with 5b5e94fd5bd8f91a89507c85487eae802f5a03c0034a9904018094c342b31060 not found: ID does not exist"
Nov 04 03:42:46.876480 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:46.871062    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j"]
Nov 04 03:42:46.876480 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:46.875152    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-b-1bf4bf-openshift-default-7fc4875485-kxt4j"]
Nov 04 03:42:47.023738 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:47.023702    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-59c311-77bc87bfc9-xvvgl"]
Nov 04 03:42:47.539189 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:47.539153    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1f315bbc-cddf-41ee-b044-92914a203e58" path="/var/lib/kubelet/pods/1f315bbc-cddf-41ee-b044-92914a203e58/volumes"
Nov 04 03:42:47.539919 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:47.539898    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="29087b39-1965-453c-9d19-61a4df9b0156" path="/var/lib/kubelet/pods/29087b39-1965-453c-9d19-61a4df9b0156/volumes"
Nov 04 03:42:48.598951 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:42:48.598917    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:42:52.024215 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:52.024160    5609 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-59c311-77bc87bfc9-xvvgl" podUID="43ac1896-22eb-4e13-889f-250eeb780886" containerName="payload-pre-processing" containerID="cri-o://a77198f873f1ffc36f522945291e11383e9cdf58da0aa3f934fcced12f7f9eab" gracePeriod=25
Nov 04 03:42:52.278791 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:52.278728    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-59c311-77bc87bfc9-xvvgl"
Nov 04 03:42:52.369619 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:52.359000    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/43ac1896-22eb-4e13-889f-250eeb780886-kube-api-access-mmtv7\" (UniqueName: \"kubernetes.io/projected/43ac1896-22eb-4e13-889f-250eeb780886-kube-api-access-mmtv7\") pod \"43ac1896-22eb-4e13-889f-250eeb780886\" (UID: \"43ac1896-22eb-4e13-889f-250eeb780886\") "
Nov 04 03:42:52.369619 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:52.359171    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/43ac1896-22eb-4e13-889f-250eeb780886-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43ac1896-22eb-4e13-889f-250eeb780886-plugins-config-volume\") pod \"43ac1896-22eb-4e13-889f-250eeb780886\" (UID: \"43ac1896-22eb-4e13-889f-250eeb780886\") "
Nov 04 03:42:52.369619 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:52.360392    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43ac1896-22eb-4e13-889f-250eeb780886-plugins-config-volume" pod "43ac1896-22eb-4e13-889f-250eeb780886" (UID: "43ac1896-22eb-4e13-889f-250eeb780886"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:42:52.383077 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:52.383038    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43ac1896-22eb-4e13-889f-250eeb780886-kube-api-access-mmtv7" pod "43ac1896-22eb-4e13-889f-250eeb780886" (UID: "43ac1896-22eb-4e13-889f-250eeb780886"). InnerVolumeSpecName "kube-api-access-mmtv7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:42:52.477423 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:52.477285    5609 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43ac1896-22eb-4e13-889f-250eeb780886-plugins-config-volume\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:52.477423 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:52.477325    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mmtv7\" (UniqueName: \"kubernetes.io/projected/43ac1896-22eb-4e13-889f-250eeb780886-kube-api-access-mmtv7\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:52.721297 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:52.721262    5609 generic.go:356] "Generic (PLEG): container finished" podID="43ac1896-22eb-4e13-889f-250eeb780886" containerID="a77198f873f1ffc36f522945291e11383e9cdf58da0aa3f934fcced12f7f9eab" exitCode=0
Nov 04 03:42:52.721484 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:52.721303    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-59c311-77bc87bfc9-xvvgl"
Nov 04 03:42:52.721484 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:52.721347    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-59c311-77bc87bfc9-xvvgl" event={"ID":"43ac1896-22eb-4e13-889f-250eeb780886","Type":"ContainerDied","Data":"a77198f873f1ffc36f522945291e11383e9cdf58da0aa3f934fcced12f7f9eab"}
Nov 04 03:42:52.721484 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:52.721416    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-59c311-77bc87bfc9-xvvgl" event={"ID":"43ac1896-22eb-4e13-889f-250eeb780886","Type":"ContainerDied","Data":"16086e2b1230855a930a4726fe3304fc725c90628edc0eb56f7f82b882803342"}
Nov 04 03:42:52.721484 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:52.721446    5609 scope.go:122] "RemoveContainer" containerID="a77198f873f1ffc36f522945291e11383e9cdf58da0aa3f934fcced12f7f9eab"
Nov 04 03:42:52.733355 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:52.733331    5609 scope.go:122] "RemoveContainer" containerID="a77198f873f1ffc36f522945291e11383e9cdf58da0aa3f934fcced12f7f9eab"
Nov 04 03:42:52.733777 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:42:52.733683    5609 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a77198f873f1ffc36f522945291e11383e9cdf58da0aa3f934fcced12f7f9eab\": container with ID starting with a77198f873f1ffc36f522945291e11383e9cdf58da0aa3f934fcced12f7f9eab not found: ID does not exist" containerID="a77198f873f1ffc36f522945291e11383e9cdf58da0aa3f934fcced12f7f9eab"
Nov 04 03:42:52.733777 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:52.733720    5609 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a77198f873f1ffc36f522945291e11383e9cdf58da0aa3f934fcced12f7f9eab"} err="failed to get container status \"a77198f873f1ffc36f522945291e11383e9cdf58da0aa3f934fcced12f7f9eab\": rpc error: code = NotFound desc = could not find container \"a77198f873f1ffc36f522945291e11383e9cdf58da0aa3f934fcced12f7f9eab\": container with ID starting with a77198f873f1ffc36f522945291e11383e9cdf58da0aa3f934fcced12f7f9eab not found: ID does not exist"
Nov 04 03:42:52.922424 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:52.918952    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-59c311-77bc87bfc9-xvvgl"]
Nov 04 03:42:52.923114 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:52.922790    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-59c311-77bc87bfc9-xvvgl"]
Nov 04 03:42:53.535042 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:53.535009    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="43ac1896-22eb-4e13-889f-250eeb780886" path="/var/lib/kubelet/pods/43ac1896-22eb-4e13-889f-250eeb780886/volumes"
Nov 04 03:42:54.305369 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:54.305333    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-50429e-7bfb595764-n2l4c"]
Nov 04 03:42:54.903713 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:54.903673    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-785569ce-k8zzt"]
Nov 04 03:42:54.904196 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:54.904182    5609 cpu_manager.go:414] "containerMap: removing container" podUID="9320f6ca-389b-4857-86b3-5336f90445ce" containerName="revoke-keys"
Nov 04 03:42:54.904240 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:54.904198    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9320f6ca-389b-4857-86b3-5336f90445ce" containerName="revoke-keys"
Nov 04 03:42:54.904240 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:54.904212    5609 cpu_manager.go:414] "containerMap: removing container" podUID="29087b39-1965-453c-9d19-61a4df9b0156" containerName="istio-proxy"
Nov 04 03:42:54.904240 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:54.904217    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="29087b39-1965-453c-9d19-61a4df9b0156" containerName="istio-proxy"
Nov 04 03:42:54.904240 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:54.904233    5609 cpu_manager.go:414] "containerMap: removing container" podUID="43ac1896-22eb-4e13-889f-250eeb780886" containerName="payload-pre-processing"
Nov 04 03:42:54.904240 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:54.904240    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="43ac1896-22eb-4e13-889f-250eeb780886" containerName="payload-pre-processing"
Nov 04 03:42:54.904385 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:54.904270    5609 cpu_manager.go:414] "containerMap: removing container" podUID="20f015bc-d781-4317-99c3-ce697c740441" containerName="istio-proxy"
Nov 04 03:42:54.904385 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:54.904275    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="20f015bc-d781-4317-99c3-ce697c740441" containerName="istio-proxy"
Nov 04 03:42:54.904385 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:54.904284    5609 cpu_manager.go:414] "containerMap: removing container" podUID="1f315bbc-cddf-41ee-b044-92914a203e58" containerName="istio-proxy"
Nov 04 03:42:54.904385 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:54.904288    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1f315bbc-cddf-41ee-b044-92914a203e58" containerName="istio-proxy"
Nov 04 03:42:54.904385 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:54.904294    5609 cpu_manager.go:414] "containerMap: removing container" podUID="1f315bbc-cddf-41ee-b044-92914a203e58" containerName="istio-proxy"
Nov 04 03:42:54.904385 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:54.904299    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1f315bbc-cddf-41ee-b044-92914a203e58" containerName="istio-proxy"
Nov 04 03:42:54.904566 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:54.904387    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="20f015bc-d781-4317-99c3-ce697c740441" containerName="istio-proxy"
Nov 04 03:42:54.904566 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:54.904394    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="29087b39-1965-453c-9d19-61a4df9b0156" containerName="istio-proxy"
Nov 04 03:42:54.904566 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:54.904413    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="1f315bbc-cddf-41ee-b044-92914a203e58" containerName="istio-proxy"
Nov 04 03:42:54.904566 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:54.904419    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="1f315bbc-cddf-41ee-b044-92914a203e58" containerName="istio-proxy"
Nov 04 03:42:54.904566 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:54.904425    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="9320f6ca-389b-4857-86b3-5336f90445ce" containerName="revoke-keys"
Nov 04 03:42:54.904566 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:54.904432    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="29087b39-1965-453c-9d19-61a4df9b0156" containerName="istio-proxy"
Nov 04 03:42:54.904566 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:54.904439    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="43ac1896-22eb-4e13-889f-250eeb780886" containerName="payload-pre-processing"
Nov 04 03:42:54.909095 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:54.909079    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-785569ce-k8zzt"
Nov 04 03:42:54.959299 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:54.958790    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 03:42:54.959299 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:54.958924    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x9cnx\""
Nov 04 03:42:54.963047 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:54.963023    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-785569ce-k8zzt"]
Nov 04 03:42:55.060195 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:55.060158    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d88r8\" (UniqueName: \"kubernetes.io/projected/30577123-d648-4623-ba41-1443be46c11e-kube-api-access-d88r8\") pod \"maas-api-revoke-sub-785569ce-k8zzt\" (UID: \"30577123-d648-4623-ba41-1443be46c11e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-785569ce-k8zzt"
Nov 04 03:42:55.060353 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:55.060218    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/30577123-d648-4623-ba41-1443be46c11e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-785569ce-k8zzt\" (UID: \"30577123-d648-4623-ba41-1443be46c11e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-785569ce-k8zzt"
Nov 04 03:42:55.060353 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:55.060327    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/30577123-d648-4623-ba41-1443be46c11e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-785569ce-k8zzt\" (UID: \"30577123-d648-4623-ba41-1443be46c11e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-785569ce-k8zzt"
Nov 04 03:42:55.161552 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:55.161469    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/30577123-d648-4623-ba41-1443be46c11e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-785569ce-k8zzt\" (UID: \"30577123-d648-4623-ba41-1443be46c11e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-785569ce-k8zzt"
Nov 04 03:42:55.161711 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:55.161602    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d88r8\" (UniqueName: \"kubernetes.io/projected/30577123-d648-4623-ba41-1443be46c11e-kube-api-access-d88r8\") pod \"maas-api-revoke-sub-785569ce-k8zzt\" (UID: \"30577123-d648-4623-ba41-1443be46c11e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-785569ce-k8zzt"
Nov 04 03:42:55.161711 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:55.161640    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/30577123-d648-4623-ba41-1443be46c11e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-785569ce-k8zzt\" (UID: \"30577123-d648-4623-ba41-1443be46c11e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-785569ce-k8zzt"
Nov 04 03:42:55.162167 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:55.162141    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/30577123-d648-4623-ba41-1443be46c11e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-785569ce-k8zzt\" (UID: \"30577123-d648-4623-ba41-1443be46c11e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-785569ce-k8zzt"
Nov 04 03:42:55.164026 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:55.164006    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/30577123-d648-4623-ba41-1443be46c11e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-785569ce-k8zzt\" (UID: \"30577123-d648-4623-ba41-1443be46c11e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-785569ce-k8zzt"
Nov 04 03:42:55.220143 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:55.220119    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d88r8\" (UniqueName: \"kubernetes.io/projected/30577123-d648-4623-ba41-1443be46c11e-kube-api-access-d88r8\") pod \"maas-api-revoke-sub-785569ce-k8zzt\" (UID: \"30577123-d648-4623-ba41-1443be46c11e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-785569ce-k8zzt"
Nov 04 03:42:55.223005 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:55.222979    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-785569ce-k8zzt"
Nov 04 03:42:55.596296 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:55.596263    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-785569ce-k8zzt"]
Nov 04 03:42:55.601189 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:42:55.601162    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30577123_d648_4623_ba41_1443be46c11e.slice/crio-ccecd6ad41c5f8aee1d0703ab9e8d0df8bca9c6ba0beddf59bb2b9a93c0babeb WatchSource:0}: Error finding container ccecd6ad41c5f8aee1d0703ab9e8d0df8bca9c6ba0beddf59bb2b9a93c0babeb: Status 404 returned error can't find the container with id ccecd6ad41c5f8aee1d0703ab9e8d0df8bca9c6ba0beddf59bb2b9a93c0babeb
Nov 04 03:42:55.783129 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:55.783065    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-785569ce-k8zzt" event={"ID":"30577123-d648-4623-ba41-1443be46c11e","Type":"ContainerStarted","Data":"ccecd6ad41c5f8aee1d0703ab9e8d0df8bca9c6ba0beddf59bb2b9a93c0babeb"}
Nov 04 03:42:56.791160 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:56.791072    5609 generic.go:356] "Generic (PLEG): container finished" podID="30577123-d648-4623-ba41-1443be46c11e" containerID="ec9b9b1ace9c5bcc1dee5f35924095e9504f310750500eb37bc79b3da6612991" exitCode=0
Nov 04 03:42:56.791574 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:56.791158    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-785569ce-k8zzt" event={"ID":"30577123-d648-4623-ba41-1443be46c11e","Type":"ContainerDied","Data":"ec9b9b1ace9c5bcc1dee5f35924095e9504f310750500eb37bc79b3da6612991"}
Nov 04 03:42:58.010696 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:58.010673    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-785569ce-k8zzt"
Nov 04 03:42:58.133012 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:58.132918    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/30577123-d648-4623-ba41-1443be46c11e-kube-api-access-d88r8\" (UniqueName: \"kubernetes.io/projected/30577123-d648-4623-ba41-1443be46c11e-kube-api-access-d88r8\") pod \"30577123-d648-4623-ba41-1443be46c11e\" (UID: \"30577123-d648-4623-ba41-1443be46c11e\") "
Nov 04 03:42:58.133183 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:58.133046    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/30577123-d648-4623-ba41-1443be46c11e-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/30577123-d648-4623-ba41-1443be46c11e-maas-api-xks-ca\") pod \"30577123-d648-4623-ba41-1443be46c11e\" (UID: \"30577123-d648-4623-ba41-1443be46c11e\") "
Nov 04 03:42:58.133183 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:58.133125    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/30577123-d648-4623-ba41-1443be46c11e-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/30577123-d648-4623-ba41-1443be46c11e-maas-api-openshift-ca\") pod \"30577123-d648-4623-ba41-1443be46c11e\" (UID: \"30577123-d648-4623-ba41-1443be46c11e\") "
Nov 04 03:42:58.133578 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:58.133548    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30577123-d648-4623-ba41-1443be46c11e-maas-api-openshift-ca" pod "30577123-d648-4623-ba41-1443be46c11e" (UID: "30577123-d648-4623-ba41-1443be46c11e"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:42:58.135133 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:58.135106    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30577123-d648-4623-ba41-1443be46c11e-kube-api-access-d88r8" pod "30577123-d648-4623-ba41-1443be46c11e" (UID: "30577123-d648-4623-ba41-1443be46c11e"). InnerVolumeSpecName "kube-api-access-d88r8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:42:58.135550 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:58.135520    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30577123-d648-4623-ba41-1443be46c11e-maas-api-xks-ca" pod "30577123-d648-4623-ba41-1443be46c11e" (UID: "30577123-d648-4623-ba41-1443be46c11e"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:42:58.234172 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:58.234137    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d88r8\" (UniqueName: \"kubernetes.io/projected/30577123-d648-4623-ba41-1443be46c11e-kube-api-access-d88r8\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:58.234172 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:58.234165    5609 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/30577123-d648-4623-ba41-1443be46c11e-maas-api-xks-ca\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:58.234172 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:58.234174    5609 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/30577123-d648-4623-ba41-1443be46c11e-maas-api-openshift-ca\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:58.810643 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:58.810612    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-785569ce-k8zzt"
Nov 04 03:42:58.810643 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:58.810624    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-785569ce-k8zzt" event={"ID":"30577123-d648-4623-ba41-1443be46c11e","Type":"ContainerDied","Data":"ccecd6ad41c5f8aee1d0703ab9e8d0df8bca9c6ba0beddf59bb2b9a93c0babeb"}
Nov 04 03:42:58.810849 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:58.810675    5609 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ccecd6ad41c5f8aee1d0703ab9e8d0df8bca9c6ba0beddf59bb2b9a93c0babeb"
Nov 04 03:42:59.306221 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:59.306183    5609 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-50429e-7bfb595764-n2l4c" podUID="295e9e28-b9ff-4dd2-b440-5dfd9190bd9a" containerName="payload-processing" containerID="cri-o://daadbd46ba630116dd54c5ed9d2c9029e1e6aabf3407335278f64b719f3ec978" gracePeriod=25
Nov 04 03:42:59.640030 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:59.640006    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-50429e-7bfb595764-n2l4c"
Nov 04 03:42:59.650312 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:59.650286    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/295e9e28-b9ff-4dd2-b440-5dfd9190bd9a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/295e9e28-b9ff-4dd2-b440-5dfd9190bd9a-plugins-config-volume\") pod \"295e9e28-b9ff-4dd2-b440-5dfd9190bd9a\" (UID: \"295e9e28-b9ff-4dd2-b440-5dfd9190bd9a\") "
Nov 04 03:42:59.650452 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:59.650438    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/295e9e28-b9ff-4dd2-b440-5dfd9190bd9a-kube-api-access-dfh9w\" (UniqueName: \"kubernetes.io/projected/295e9e28-b9ff-4dd2-b440-5dfd9190bd9a-kube-api-access-dfh9w\") pod \"295e9e28-b9ff-4dd2-b440-5dfd9190bd9a\" (UID: \"295e9e28-b9ff-4dd2-b440-5dfd9190bd9a\") "
Nov 04 03:42:59.650635 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:59.650615    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/295e9e28-b9ff-4dd2-b440-5dfd9190bd9a-plugins-config-volume" pod "295e9e28-b9ff-4dd2-b440-5dfd9190bd9a" (UID: "295e9e28-b9ff-4dd2-b440-5dfd9190bd9a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:42:59.650762 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:59.650747    5609 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/295e9e28-b9ff-4dd2-b440-5dfd9190bd9a-plugins-config-volume\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:59.652332 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:59.652309    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/295e9e28-b9ff-4dd2-b440-5dfd9190bd9a-kube-api-access-dfh9w" pod "295e9e28-b9ff-4dd2-b440-5dfd9190bd9a" (UID: "295e9e28-b9ff-4dd2-b440-5dfd9190bd9a"). InnerVolumeSpecName "kube-api-access-dfh9w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:42:59.751658 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:59.751627    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dfh9w\" (UniqueName: \"kubernetes.io/projected/295e9e28-b9ff-4dd2-b440-5dfd9190bd9a-kube-api-access-dfh9w\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:59.822167 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:59.822120    5609 generic.go:356] "Generic (PLEG): container finished" podID="295e9e28-b9ff-4dd2-b440-5dfd9190bd9a" containerID="daadbd46ba630116dd54c5ed9d2c9029e1e6aabf3407335278f64b719f3ec978" exitCode=0
Nov 04 03:42:59.822328 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:59.822285    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-50429e-7bfb595764-n2l4c" event={"ID":"295e9e28-b9ff-4dd2-b440-5dfd9190bd9a","Type":"ContainerDied","Data":"daadbd46ba630116dd54c5ed9d2c9029e1e6aabf3407335278f64b719f3ec978"}
Nov 04 03:42:59.822440 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:59.822367    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-50429e-7bfb595764-n2l4c" event={"ID":"295e9e28-b9ff-4dd2-b440-5dfd9190bd9a","Type":"ContainerDied","Data":"41cbc06852a7e28e7ff921cb6d5ba4434e3476292e3360beeac6df74b0b34755"}
Nov 04 03:42:59.822440 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:59.822422    5609 scope.go:122] "RemoveContainer" containerID="daadbd46ba630116dd54c5ed9d2c9029e1e6aabf3407335278f64b719f3ec978"
Nov 04 03:42:59.822798 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:59.822618    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-50429e-7bfb595764-n2l4c"
Nov 04 03:42:59.836687 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:59.836661    5609 scope.go:122] "RemoveContainer" containerID="daadbd46ba630116dd54c5ed9d2c9029e1e6aabf3407335278f64b719f3ec978"
Nov 04 03:42:59.837005 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:42:59.836985    5609 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"daadbd46ba630116dd54c5ed9d2c9029e1e6aabf3407335278f64b719f3ec978\": container with ID starting with daadbd46ba630116dd54c5ed9d2c9029e1e6aabf3407335278f64b719f3ec978 not found: ID does not exist" containerID="daadbd46ba630116dd54c5ed9d2c9029e1e6aabf3407335278f64b719f3ec978"
Nov 04 03:42:59.837058 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:59.837013    5609 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"daadbd46ba630116dd54c5ed9d2c9029e1e6aabf3407335278f64b719f3ec978"} err="failed to get container status \"daadbd46ba630116dd54c5ed9d2c9029e1e6aabf3407335278f64b719f3ec978\": rpc error: code = NotFound desc = could not find container \"daadbd46ba630116dd54c5ed9d2c9029e1e6aabf3407335278f64b719f3ec978\": container with ID starting with daadbd46ba630116dd54c5ed9d2c9029e1e6aabf3407335278f64b719f3ec978 not found: ID does not exist"
Nov 04 03:42:59.980068 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:59.980025    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-50429e-7bfb595764-n2l4c"]
Nov 04 03:42:59.984550 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:42:59.984523    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-50429e-7bfb595764-n2l4c"]
Nov 04 03:43:01.534347 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:01.534314    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="295e9e28-b9ff-4dd2-b440-5dfd9190bd9a" path="/var/lib/kubelet/pods/295e9e28-b9ff-4dd2-b440-5dfd9190bd9a/volumes"
Nov 04 03:43:02.881090 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:02.878873    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c"]
Nov 04 03:43:02.881090 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:02.879192    5609 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c" podUID="5e6b2f82-51f7-49e4-b952-7a0861e35889" containerName="istio-proxy" containerID="cri-o://43726a00aff279cf31edf98af79ed074a9ed86a859b55b9a72f6cc113ab2bb3b" gracePeriod=30
Nov 04 03:43:04.756445 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:04.756408    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-82e703ce-slsch"]
Nov 04 03:43:04.756991 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:04.756972    5609 cpu_manager.go:414] "containerMap: removing container" podUID="29087b39-1965-453c-9d19-61a4df9b0156" containerName="istio-proxy"
Nov 04 03:43:04.757044 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:04.756994    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="29087b39-1965-453c-9d19-61a4df9b0156" containerName="istio-proxy"
Nov 04 03:43:04.757076 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:04.757046    5609 cpu_manager.go:414] "containerMap: removing container" podUID="295e9e28-b9ff-4dd2-b440-5dfd9190bd9a" containerName="payload-processing"
Nov 04 03:43:04.757076 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:04.757052    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="295e9e28-b9ff-4dd2-b440-5dfd9190bd9a" containerName="payload-processing"
Nov 04 03:43:04.757076 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:04.757061    5609 cpu_manager.go:414] "containerMap: removing container" podUID="30577123-d648-4623-ba41-1443be46c11e" containerName="revoke-keys"
Nov 04 03:43:04.757076 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:04.757067    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="30577123-d648-4623-ba41-1443be46c11e" containerName="revoke-keys"
Nov 04 03:43:04.757189 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:04.757147    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="295e9e28-b9ff-4dd2-b440-5dfd9190bd9a" containerName="payload-processing"
Nov 04 03:43:04.757189 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:04.757156    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="30577123-d648-4623-ba41-1443be46c11e" containerName="revoke-keys"
Nov 04 03:43:04.761980 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:04.761961    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-82e703ce-slsch"
Nov 04 03:43:04.809953 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:04.809920    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwg8d\" (UniqueName: \"kubernetes.io/projected/98eb52e2-860b-4b49-9cc9-7b971fdb5224-kube-api-access-zwg8d\") pod \"maas-api-revoke-sub-82e703ce-slsch\" (UID: \"98eb52e2-860b-4b49-9cc9-7b971fdb5224\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-82e703ce-slsch"
Nov 04 03:43:04.810103 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:04.809991    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/98eb52e2-860b-4b49-9cc9-7b971fdb5224-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-82e703ce-slsch\" (UID: \"98eb52e2-860b-4b49-9cc9-7b971fdb5224\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-82e703ce-slsch"
Nov 04 03:43:04.810150 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:04.810105    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/98eb52e2-860b-4b49-9cc9-7b971fdb5224-maas-api-xks-ca\") pod \"maas-api-revoke-sub-82e703ce-slsch\" (UID: \"98eb52e2-860b-4b49-9cc9-7b971fdb5224\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-82e703ce-slsch"
Nov 04 03:43:04.811380 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:04.811362    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x9cnx\""
Nov 04 03:43:04.811509 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:04.811494    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 03:43:04.899833 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:04.899797    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-82e703ce-slsch"]
Nov 04 03:43:04.911452 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:04.911379    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/98eb52e2-860b-4b49-9cc9-7b971fdb5224-maas-api-xks-ca\") pod \"maas-api-revoke-sub-82e703ce-slsch\" (UID: \"98eb52e2-860b-4b49-9cc9-7b971fdb5224\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-82e703ce-slsch"
Nov 04 03:43:04.911642 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:04.911497    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwg8d\" (UniqueName: \"kubernetes.io/projected/98eb52e2-860b-4b49-9cc9-7b971fdb5224-kube-api-access-zwg8d\") pod \"maas-api-revoke-sub-82e703ce-slsch\" (UID: \"98eb52e2-860b-4b49-9cc9-7b971fdb5224\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-82e703ce-slsch"
Nov 04 03:43:04.911642 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:04.911558    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/98eb52e2-860b-4b49-9cc9-7b971fdb5224-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-82e703ce-slsch\" (UID: \"98eb52e2-860b-4b49-9cc9-7b971fdb5224\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-82e703ce-slsch"
Nov 04 03:43:04.912190 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:04.912169    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/98eb52e2-860b-4b49-9cc9-7b971fdb5224-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-82e703ce-slsch\" (UID: \"98eb52e2-860b-4b49-9cc9-7b971fdb5224\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-82e703ce-slsch"
Nov 04 03:43:04.913833 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:04.913808    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/98eb52e2-860b-4b49-9cc9-7b971fdb5224-maas-api-xks-ca\") pod \"maas-api-revoke-sub-82e703ce-slsch\" (UID: \"98eb52e2-860b-4b49-9cc9-7b971fdb5224\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-82e703ce-slsch"
Nov 04 03:43:04.970331 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:04.970302    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwg8d\" (UniqueName: \"kubernetes.io/projected/98eb52e2-860b-4b49-9cc9-7b971fdb5224-kube-api-access-zwg8d\") pod \"maas-api-revoke-sub-82e703ce-slsch\" (UID: \"98eb52e2-860b-4b49-9cc9-7b971fdb5224\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-82e703ce-slsch"
Nov 04 03:43:05.076448 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:05.076350    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-82e703ce-slsch"
Nov 04 03:43:05.472919 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:05.472884    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-82e703ce-slsch"]
Nov 04 03:43:05.477474 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:43:05.477443    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98eb52e2_860b_4b49_9cc9_7b971fdb5224.slice/crio-c7961a828500d231d16140ead97f6c27644a59b85466baa2532845129638e3e6 WatchSource:0}: Error finding container c7961a828500d231d16140ead97f6c27644a59b85466baa2532845129638e3e6: Status 404 returned error can't find the container with id c7961a828500d231d16140ead97f6c27644a59b85466baa2532845129638e3e6
Nov 04 03:43:05.916267 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:05.916231    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-82e703ce-slsch" event={"ID":"98eb52e2-860b-4b49-9cc9-7b971fdb5224","Type":"ContainerStarted","Data":"0b913e2606895e69b7677ea3bc704589fd8e815b23a9c2e5e909e740d4cb80d2"}
Nov 04 03:43:05.916691 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:05.916300    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-82e703ce-slsch" event={"ID":"98eb52e2-860b-4b49-9cc9-7b971fdb5224","Type":"ContainerStarted","Data":"c7961a828500d231d16140ead97f6c27644a59b85466baa2532845129638e3e6"}
Nov 04 03:43:06.027817 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:06.027767    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-82e703ce-slsch" podStartSLOduration=2.027751628 podStartE2EDuration="2.027751628s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:43:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:43:06.025635352 +0000 UTC m=+1528.602288267" watchObservedRunningTime="2026-11-04 03:43:06.027751628 +0000 UTC m=+1528.604404547"
Nov 04 03:43:06.924915 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:06.924882    5609 generic.go:356] "Generic (PLEG): container finished" podID="98eb52e2-860b-4b49-9cc9-7b971fdb5224" containerID="0b913e2606895e69b7677ea3bc704589fd8e815b23a9c2e5e909e740d4cb80d2" exitCode=0
Nov 04 03:43:06.925334 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:06.924968    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-82e703ce-slsch" event={"ID":"98eb52e2-860b-4b49-9cc9-7b971fdb5224","Type":"ContainerDied","Data":"0b913e2606895e69b7677ea3bc704589fd8e815b23a9c2e5e909e740d4cb80d2"}
Nov 04 03:43:08.163506 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:08.162208    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-82e703ce-slsch"
Nov 04 03:43:08.315018 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:08.311515    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c"
Nov 04 03:43:08.360336 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:08.360305    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5e6b2f82-51f7-49e4-b952-7a0861e35889-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e6b2f82-51f7-49e4-b952-7a0861e35889-istiod-ca-cert\") pod \"5e6b2f82-51f7-49e4-b952-7a0861e35889\" (UID: \"5e6b2f82-51f7-49e4-b952-7a0861e35889\") "
Nov 04 03:43:08.360551 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:08.360347    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/98eb52e2-860b-4b49-9cc9-7b971fdb5224-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/98eb52e2-860b-4b49-9cc9-7b971fdb5224-maas-api-openshift-ca\") pod \"98eb52e2-860b-4b49-9cc9-7b971fdb5224\" (UID: \"98eb52e2-860b-4b49-9cc9-7b971fdb5224\") "
Nov 04 03:43:08.360551 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:08.360379    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5e6b2f82-51f7-49e4-b952-7a0861e35889-istio-token\" (UniqueName: \"kubernetes.io/projected/5e6b2f82-51f7-49e4-b952-7a0861e35889-istio-token\") pod \"5e6b2f82-51f7-49e4-b952-7a0861e35889\" (UID: \"5e6b2f82-51f7-49e4-b952-7a0861e35889\") "
Nov 04 03:43:08.360551 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:08.360440    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/98eb52e2-860b-4b49-9cc9-7b971fdb5224-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/98eb52e2-860b-4b49-9cc9-7b971fdb5224-maas-api-xks-ca\") pod \"98eb52e2-860b-4b49-9cc9-7b971fdb5224\" (UID: \"98eb52e2-860b-4b49-9cc9-7b971fdb5224\") "
Nov 04 03:43:08.360551 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:08.360474    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e6b2f82-51f7-49e4-b952-7a0861e35889-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e6b2f82-51f7-49e4-b952-7a0861e35889-credential-socket\") pod \"5e6b2f82-51f7-49e4-b952-7a0861e35889\" (UID: \"5e6b2f82-51f7-49e4-b952-7a0861e35889\") "
Nov 04 03:43:08.360551 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:08.360504    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e6b2f82-51f7-49e4-b952-7a0861e35889-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e6b2f82-51f7-49e4-b952-7a0861e35889-workload-certs\") pod \"5e6b2f82-51f7-49e4-b952-7a0861e35889\" (UID: \"5e6b2f82-51f7-49e4-b952-7a0861e35889\") "
Nov 04 03:43:08.360824 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:08.360571    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/98eb52e2-860b-4b49-9cc9-7b971fdb5224-kube-api-access-zwg8d\" (UniqueName: \"kubernetes.io/projected/98eb52e2-860b-4b49-9cc9-7b971fdb5224-kube-api-access-zwg8d\") pod \"98eb52e2-860b-4b49-9cc9-7b971fdb5224\" (UID: \"98eb52e2-860b-4b49-9cc9-7b971fdb5224\") "
Nov 04 03:43:08.360824 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:08.360651    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5e6b2f82-51f7-49e4-b952-7a0861e35889-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e6b2f82-51f7-49e4-b952-7a0861e35889-istio-podinfo\") pod \"5e6b2f82-51f7-49e4-b952-7a0861e35889\" (UID: \"5e6b2f82-51f7-49e4-b952-7a0861e35889\") "
Nov 04 03:43:08.360824 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:08.360691    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e6b2f82-51f7-49e4-b952-7a0861e35889-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e6b2f82-51f7-49e4-b952-7a0861e35889-workload-socket\") pod \"5e6b2f82-51f7-49e4-b952-7a0861e35889\" (UID: \"5e6b2f82-51f7-49e4-b952-7a0861e35889\") "
Nov 04 03:43:08.360824 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:08.360739    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e6b2f82-51f7-49e4-b952-7a0861e35889-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e6b2f82-51f7-49e4-b952-7a0861e35889-istio-data\") pod \"5e6b2f82-51f7-49e4-b952-7a0861e35889\" (UID: \"5e6b2f82-51f7-49e4-b952-7a0861e35889\") "
Nov 04 03:43:08.360824 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:08.360795    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e6b2f82-51f7-49e4-b952-7a0861e35889-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e6b2f82-51f7-49e4-b952-7a0861e35889-istio-envoy\") pod \"5e6b2f82-51f7-49e4-b952-7a0861e35889\" (UID: \"5e6b2f82-51f7-49e4-b952-7a0861e35889\") "
Nov 04 03:43:08.360824 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:08.360822    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5e6b2f82-51f7-49e4-b952-7a0861e35889-kube-api-access-mkqk2\" (UniqueName: \"kubernetes.io/projected/5e6b2f82-51f7-49e4-b952-7a0861e35889-kube-api-access-mkqk2\") pod \"5e6b2f82-51f7-49e4-b952-7a0861e35889\" (UID: \"5e6b2f82-51f7-49e4-b952-7a0861e35889\") "
Nov 04 03:43:08.361296 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:08.361265    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98eb52e2-860b-4b49-9cc9-7b971fdb5224-maas-api-openshift-ca" pod "98eb52e2-860b-4b49-9cc9-7b971fdb5224" (UID: "98eb52e2-860b-4b49-9cc9-7b971fdb5224"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:43:08.361734 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:08.361668    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e6b2f82-51f7-49e4-b952-7a0861e35889-workload-socket" pod "5e6b2f82-51f7-49e4-b952-7a0861e35889" (UID: "5e6b2f82-51f7-49e4-b952-7a0861e35889"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:43:08.361831 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:08.361724    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e6b2f82-51f7-49e4-b952-7a0861e35889-credential-socket" pod "5e6b2f82-51f7-49e4-b952-7a0861e35889" (UID: "5e6b2f82-51f7-49e4-b952-7a0861e35889"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:43:08.361937 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:08.361912    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e6b2f82-51f7-49e4-b952-7a0861e35889-istio-data" pod "5e6b2f82-51f7-49e4-b952-7a0861e35889" (UID: "5e6b2f82-51f7-49e4-b952-7a0861e35889"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:43:08.362047 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:08.362025    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e6b2f82-51f7-49e4-b952-7a0861e35889-workload-certs" pod "5e6b2f82-51f7-49e4-b952-7a0861e35889" (UID: "5e6b2f82-51f7-49e4-b952-7a0861e35889"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:43:08.362193 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:08.362168    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e6b2f82-51f7-49e4-b952-7a0861e35889-istiod-ca-cert" pod "5e6b2f82-51f7-49e4-b952-7a0861e35889" (UID: "5e6b2f82-51f7-49e4-b952-7a0861e35889"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:43:08.364060 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:08.364026    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e6b2f82-51f7-49e4-b952-7a0861e35889-istio-envoy" pod "5e6b2f82-51f7-49e4-b952-7a0861e35889" (UID: "5e6b2f82-51f7-49e4-b952-7a0861e35889"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:43:08.365096 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:08.365067    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98eb52e2-860b-4b49-9cc9-7b971fdb5224-maas-api-xks-ca" pod "98eb52e2-860b-4b49-9cc9-7b971fdb5224" (UID: "98eb52e2-860b-4b49-9cc9-7b971fdb5224"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:43:08.365324 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:08.365270    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98eb52e2-860b-4b49-9cc9-7b971fdb5224-kube-api-access-zwg8d" pod "98eb52e2-860b-4b49-9cc9-7b971fdb5224" (UID: "98eb52e2-860b-4b49-9cc9-7b971fdb5224"). InnerVolumeSpecName "kube-api-access-zwg8d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:43:08.365324 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:08.365292    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e6b2f82-51f7-49e4-b952-7a0861e35889-kube-api-access-mkqk2" pod "5e6b2f82-51f7-49e4-b952-7a0861e35889" (UID: "5e6b2f82-51f7-49e4-b952-7a0861e35889"). InnerVolumeSpecName "kube-api-access-mkqk2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:43:08.366669 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:08.366633    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e6b2f82-51f7-49e4-b952-7a0861e35889-istio-token" pod "5e6b2f82-51f7-49e4-b952-7a0861e35889" (UID: "5e6b2f82-51f7-49e4-b952-7a0861e35889"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:43:08.368346 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:08.368318    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5e6b2f82-51f7-49e4-b952-7a0861e35889-istio-podinfo" pod "5e6b2f82-51f7-49e4-b952-7a0861e35889" (UID: "5e6b2f82-51f7-49e4-b952-7a0861e35889"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:43:08.474304 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:08.473654    5609 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e6b2f82-51f7-49e4-b952-7a0861e35889-istiod-ca-cert\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:08.474304 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:08.473698    5609 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/98eb52e2-860b-4b49-9cc9-7b971fdb5224-maas-api-openshift-ca\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:08.474304 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:08.473715    5609 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e6b2f82-51f7-49e4-b952-7a0861e35889-istio-token\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:08.474304 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:08.473730    5609 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/98eb52e2-860b-4b49-9cc9-7b971fdb5224-maas-api-xks-ca\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:08.474304 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:08.473744    5609 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e6b2f82-51f7-49e4-b952-7a0861e35889-credential-socket\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:08.474304 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:08.473759    5609 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e6b2f82-51f7-49e4-b952-7a0861e35889-workload-certs\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:08.474304 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:08.473772    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zwg8d\" (UniqueName: \"kubernetes.io/projected/98eb52e2-860b-4b49-9cc9-7b971fdb5224-kube-api-access-zwg8d\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:08.474304 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:08.473786    5609 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e6b2f82-51f7-49e4-b952-7a0861e35889-istio-podinfo\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:08.474304 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:08.473799    5609 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e6b2f82-51f7-49e4-b952-7a0861e35889-workload-socket\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:08.474304 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:08.473812    5609 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e6b2f82-51f7-49e4-b952-7a0861e35889-istio-data\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:08.474304 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:08.473824    5609 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e6b2f82-51f7-49e4-b952-7a0861e35889-istio-envoy\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:08.474304 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:08.473838    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mkqk2\" (UniqueName: \"kubernetes.io/projected/5e6b2f82-51f7-49e4-b952-7a0861e35889-kube-api-access-mkqk2\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:08.971333 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:08.966575    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-82e703ce-slsch" event={"ID":"98eb52e2-860b-4b49-9cc9-7b971fdb5224","Type":"ContainerDied","Data":"c7961a828500d231d16140ead97f6c27644a59b85466baa2532845129638e3e6"}
Nov 04 03:43:08.971333 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:08.966653    5609 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c7961a828500d231d16140ead97f6c27644a59b85466baa2532845129638e3e6"
Nov 04 03:43:08.971333 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:08.966737    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-82e703ce-slsch"
Nov 04 03:43:08.972315 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:08.972282    5609 generic.go:356] "Generic (PLEG): container finished" podID="5e6b2f82-51f7-49e4-b952-7a0861e35889" containerID="43726a00aff279cf31edf98af79ed074a9ed86a859b55b9a72f6cc113ab2bb3b" exitCode=0
Nov 04 03:43:08.972477 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:08.972387    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c" event={"ID":"5e6b2f82-51f7-49e4-b952-7a0861e35889","Type":"ContainerDied","Data":"43726a00aff279cf31edf98af79ed074a9ed86a859b55b9a72f6cc113ab2bb3b"}
Nov 04 03:43:08.972477 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:08.972461    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c" event={"ID":"5e6b2f82-51f7-49e4-b952-7a0861e35889","Type":"ContainerDied","Data":"10aa41ddc5533ea407bad7817f4a40b280db638880068d258f294b70f410c0f3"}
Nov 04 03:43:08.972477 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:08.972466    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c"
Nov 04 03:43:08.972617 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:08.972492    5609 scope.go:122] "RemoveContainer" containerID="43726a00aff279cf31edf98af79ed074a9ed86a859b55b9a72f6cc113ab2bb3b"
Nov 04 03:43:09.002636 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:08.984628    5609 scope.go:122] "RemoveContainer" containerID="64a5416ece825642bebb2ea435c8bfc743e4c4e9133eeab52d306482b9545b1a"
Nov 04 03:43:09.035117 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:09.030781    5609 scope.go:122] "RemoveContainer" containerID="43726a00aff279cf31edf98af79ed074a9ed86a859b55b9a72f6cc113ab2bb3b"
Nov 04 03:43:09.035117 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:43:09.031223    5609 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43726a00aff279cf31edf98af79ed074a9ed86a859b55b9a72f6cc113ab2bb3b\": container with ID starting with 43726a00aff279cf31edf98af79ed074a9ed86a859b55b9a72f6cc113ab2bb3b not found: ID does not exist" containerID="43726a00aff279cf31edf98af79ed074a9ed86a859b55b9a72f6cc113ab2bb3b"
Nov 04 03:43:09.035117 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:09.031260    5609 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43726a00aff279cf31edf98af79ed074a9ed86a859b55b9a72f6cc113ab2bb3b"} err="failed to get container status \"43726a00aff279cf31edf98af79ed074a9ed86a859b55b9a72f6cc113ab2bb3b\": rpc error: code = NotFound desc = could not find container \"43726a00aff279cf31edf98af79ed074a9ed86a859b55b9a72f6cc113ab2bb3b\": container with ID starting with 43726a00aff279cf31edf98af79ed074a9ed86a859b55b9a72f6cc113ab2bb3b not found: ID does not exist"
Nov 04 03:43:09.035117 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:09.031285    5609 scope.go:122] "RemoveContainer" containerID="64a5416ece825642bebb2ea435c8bfc743e4c4e9133eeab52d306482b9545b1a"
Nov 04 03:43:09.035117 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:43:09.031624    5609 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64a5416ece825642bebb2ea435c8bfc743e4c4e9133eeab52d306482b9545b1a\": container with ID starting with 64a5416ece825642bebb2ea435c8bfc743e4c4e9133eeab52d306482b9545b1a not found: ID does not exist" containerID="64a5416ece825642bebb2ea435c8bfc743e4c4e9133eeab52d306482b9545b1a"
Nov 04 03:43:09.035117 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:09.031649    5609 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64a5416ece825642bebb2ea435c8bfc743e4c4e9133eeab52d306482b9545b1a"} err="failed to get container status \"64a5416ece825642bebb2ea435c8bfc743e4c4e9133eeab52d306482b9545b1a\": rpc error: code = NotFound desc = could not find container \"64a5416ece825642bebb2ea435c8bfc743e4c4e9133eeab52d306482b9545b1a\": container with ID starting with 64a5416ece825642bebb2ea435c8bfc743e4c4e9133eeab52d306482b9545b1a not found: ID does not exist"
Nov 04 03:43:09.204425 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:09.198143    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c"]
Nov 04 03:43:09.265456 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:09.247074    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-b-59c311-openshift-default-5dbbb75c65-bf52c"]
Nov 04 03:43:09.419426 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:09.415853    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-df4cf9975-hk75w"]
Nov 04 03:43:09.419426 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:09.416653    5609 cpu_manager.go:414] "containerMap: removing container" podUID="5e6b2f82-51f7-49e4-b952-7a0861e35889" containerName="istio-proxy"
Nov 04 03:43:09.419426 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:09.416676    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5e6b2f82-51f7-49e4-b952-7a0861e35889" containerName="istio-proxy"
Nov 04 03:43:09.419426 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:09.416693    5609 cpu_manager.go:414] "containerMap: removing container" podUID="98eb52e2-860b-4b49-9cc9-7b971fdb5224" containerName="revoke-keys"
Nov 04 03:43:09.419426 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:09.416701    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="98eb52e2-860b-4b49-9cc9-7b971fdb5224" containerName="revoke-keys"
Nov 04 03:43:09.419426 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:09.416725    5609 cpu_manager.go:414] "containerMap: removing container" podUID="5e6b2f82-51f7-49e4-b952-7a0861e35889" containerName="istio-proxy"
Nov 04 03:43:09.419426 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:09.416733    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5e6b2f82-51f7-49e4-b952-7a0861e35889" containerName="istio-proxy"
Nov 04 03:43:09.419426 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:09.416891    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="5e6b2f82-51f7-49e4-b952-7a0861e35889" containerName="istio-proxy"
Nov 04 03:43:09.419426 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:09.416908    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="98eb52e2-860b-4b49-9cc9-7b971fdb5224" containerName="revoke-keys"
Nov 04 03:43:09.419426 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:09.416920    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="5e6b2f82-51f7-49e4-b952-7a0861e35889" containerName="istio-proxy"
Nov 04 03:43:09.473468 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:09.470616    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-df4cf9975-hk75w"]
Nov 04 03:43:09.473468 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:09.470749    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-df4cf9975-hk75w"
Nov 04 03:43:09.497654 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:09.488823    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d-provider-credentials\") pod \"payload-processing-external-model-df4cf9975-hk75w\" (UID: \"0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d\") " pod="models-as-a-service/payload-processing-external-model-df4cf9975-hk75w"
Nov 04 03:43:09.497654 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:09.488881    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d-routing-overlay-volume\") pod \"payload-processing-external-model-df4cf9975-hk75w\" (UID: \"0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d\") " pod="models-as-a-service/payload-processing-external-model-df4cf9975-hk75w"
Nov 04 03:43:09.497654 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:09.488950    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d-plugins-config-volume\") pod \"payload-processing-external-model-df4cf9975-hk75w\" (UID: \"0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d\") " pod="models-as-a-service/payload-processing-external-model-df4cf9975-hk75w"
Nov 04 03:43:09.497654 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:09.489000    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsvsr\" (UniqueName: \"kubernetes.io/projected/0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d-kube-api-access-fsvsr\") pod \"payload-processing-external-model-df4cf9975-hk75w\" (UID: \"0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d\") " pod="models-as-a-service/payload-processing-external-model-df4cf9975-hk75w"
Nov 04 03:43:09.565459 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:09.564985    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"kube-root-ca.crt\""
Nov 04 03:43:09.566575 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:09.566128    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"routing-overlay\""
Nov 04 03:43:09.566575 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:09.566230    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-plugins\""
Nov 04 03:43:09.566575 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:09.566305    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-8b97550813bf\""
Nov 04 03:43:09.566575 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:09.566361    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"openshift-service-ca.crt\""
Nov 04 03:43:09.567488 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:09.567063    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-dockercfg-mlxkm\""
Nov 04 03:43:09.567488 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:09.567150    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-3d8f44c40ee8\""
Nov 04 03:43:09.597440 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:09.592018    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d-plugins-config-volume\") pod \"payload-processing-external-model-df4cf9975-hk75w\" (UID: \"0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d\") " pod="models-as-a-service/payload-processing-external-model-df4cf9975-hk75w"
Nov 04 03:43:09.597440 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:09.592103    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsvsr\" (UniqueName: \"kubernetes.io/projected/0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d-kube-api-access-fsvsr\") pod \"payload-processing-external-model-df4cf9975-hk75w\" (UID: \"0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d\") " pod="models-as-a-service/payload-processing-external-model-df4cf9975-hk75w"
Nov 04 03:43:09.597440 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:09.592203    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d-provider-credentials\") pod \"payload-processing-external-model-df4cf9975-hk75w\" (UID: \"0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d\") " pod="models-as-a-service/payload-processing-external-model-df4cf9975-hk75w"
Nov 04 03:43:09.597440 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:09.592257    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d-routing-overlay-volume\") pod \"payload-processing-external-model-df4cf9975-hk75w\" (UID: \"0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d\") " pod="models-as-a-service/payload-processing-external-model-df4cf9975-hk75w"
Nov 04 03:43:09.597440 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:09.593114    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d-routing-overlay-volume\") pod \"payload-processing-external-model-df4cf9975-hk75w\" (UID: \"0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d\") " pod="models-as-a-service/payload-processing-external-model-df4cf9975-hk75w"
Nov 04 03:43:09.607796 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:09.607734    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d-plugins-config-volume\") pod \"payload-processing-external-model-df4cf9975-hk75w\" (UID: \"0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d\") " pod="models-as-a-service/payload-processing-external-model-df4cf9975-hk75w"
Nov 04 03:43:09.619597 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:09.619570    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5e6b2f82-51f7-49e4-b952-7a0861e35889" path="/var/lib/kubelet/pods/5e6b2f82-51f7-49e4-b952-7a0861e35889/volumes"
Nov 04 03:43:09.625088 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:09.625043    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d-provider-credentials\") pod \"payload-processing-external-model-df4cf9975-hk75w\" (UID: \"0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d\") " pod="models-as-a-service/payload-processing-external-model-df4cf9975-hk75w"
Nov 04 03:43:09.671418 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:09.669900    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsvsr\" (UniqueName: \"kubernetes.io/projected/0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d-kube-api-access-fsvsr\") pod \"payload-processing-external-model-df4cf9975-hk75w\" (UID: \"0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d\") " pod="models-as-a-service/payload-processing-external-model-df4cf9975-hk75w"
Nov 04 03:43:09.880748 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:09.880652    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-df4cf9975-hk75w"
Nov 04 03:43:10.482830 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:10.482771    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-df4cf9975-hk75w"]
Nov 04 03:43:10.992115 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:10.992021    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-df4cf9975-hk75w" event={"ID":"0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d","Type":"ContainerStarted","Data":"95b5da072706d8be768c408c6bc1c096a94ce103b5c40952474ef3f0f38e845c"}
Nov 04 03:43:10.992115 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:10.992078    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-df4cf9975-hk75w" event={"ID":"0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d","Type":"ContainerStarted","Data":"2d8642dae14bec723bbb480618fcfd639d61347c9883476023f37ef6d18feae8"}
Nov 04 03:43:10.992115 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:10.992105    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-df4cf9975-hk75w"
Nov 04 03:43:11.104259 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:11.104207    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-df4cf9975-hk75w" podStartSLOduration=2.104191142 podStartE2EDuration="2.104191142s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:43:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:43:11.102985098 +0000 UTC m=+1533.679638012" watchObservedRunningTime="2026-11-04 03:43:11.104191142 +0000 UTC m=+1533.680844057"
Nov 04 03:43:18.561642 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:43:18.561550    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:43:21.472427 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:21.472379    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-82c724-6d4b6d5bb8-8cdsz"]
Nov 04 03:43:21.517777 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:21.517739    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2c87d0f1-f6pvr"]
Nov 04 03:43:21.523156 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:21.523135    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2c87d0f1-f6pvr"
Nov 04 03:43:21.575173 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:21.575146    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 03:43:21.575425 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:21.575375    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x9cnx\""
Nov 04 03:43:21.577667 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:21.577644    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2c87d0f1-f6pvr"]
Nov 04 03:43:21.587859 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:21.587828    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-82c724-76bcd64bff-94mnv"]
Nov 04 03:43:21.605026 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:21.604995    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fc5519c5-c7a9-40e0-8eff-b4d0b8887f32-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2c87d0f1-f6pvr\" (UID: \"fc5519c5-c7a9-40e0-8eff-b4d0b8887f32\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2c87d0f1-f6pvr"
Nov 04 03:43:21.605183 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:21.605034    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fc5519c5-c7a9-40e0-8eff-b4d0b8887f32-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2c87d0f1-f6pvr\" (UID: \"fc5519c5-c7a9-40e0-8eff-b4d0b8887f32\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2c87d0f1-f6pvr"
Nov 04 03:43:21.605226 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:21.605193    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h86dk\" (UniqueName: \"kubernetes.io/projected/fc5519c5-c7a9-40e0-8eff-b4d0b8887f32-kube-api-access-h86dk\") pod \"maas-api-revoke-sub-2c87d0f1-f6pvr\" (UID: \"fc5519c5-c7a9-40e0-8eff-b4d0b8887f32\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2c87d0f1-f6pvr"
Nov 04 03:43:21.706342 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:21.706306    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fc5519c5-c7a9-40e0-8eff-b4d0b8887f32-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2c87d0f1-f6pvr\" (UID: \"fc5519c5-c7a9-40e0-8eff-b4d0b8887f32\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2c87d0f1-f6pvr"
Nov 04 03:43:21.706633 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:21.706353    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fc5519c5-c7a9-40e0-8eff-b4d0b8887f32-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2c87d0f1-f6pvr\" (UID: \"fc5519c5-c7a9-40e0-8eff-b4d0b8887f32\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2c87d0f1-f6pvr"
Nov 04 03:43:21.706633 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:21.706559    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h86dk\" (UniqueName: \"kubernetes.io/projected/fc5519c5-c7a9-40e0-8eff-b4d0b8887f32-kube-api-access-h86dk\") pod \"maas-api-revoke-sub-2c87d0f1-f6pvr\" (UID: \"fc5519c5-c7a9-40e0-8eff-b4d0b8887f32\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2c87d0f1-f6pvr"
Nov 04 03:43:21.707196 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:21.707169    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fc5519c5-c7a9-40e0-8eff-b4d0b8887f32-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2c87d0f1-f6pvr\" (UID: \"fc5519c5-c7a9-40e0-8eff-b4d0b8887f32\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2c87d0f1-f6pvr"
Nov 04 03:43:21.708847 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:21.708823    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fc5519c5-c7a9-40e0-8eff-b4d0b8887f32-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2c87d0f1-f6pvr\" (UID: \"fc5519c5-c7a9-40e0-8eff-b4d0b8887f32\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2c87d0f1-f6pvr"
Nov 04 03:43:21.769385 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:21.769320    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h86dk\" (UniqueName: \"kubernetes.io/projected/fc5519c5-c7a9-40e0-8eff-b4d0b8887f32-kube-api-access-h86dk\") pod \"maas-api-revoke-sub-2c87d0f1-f6pvr\" (UID: \"fc5519c5-c7a9-40e0-8eff-b4d0b8887f32\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2c87d0f1-f6pvr"
Nov 04 03:43:21.839236 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:21.839203    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2c87d0f1-f6pvr"
Nov 04 03:43:22.001875 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:22.001847    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="models-as-a-service/payload-processing-external-model-df4cf9975-hk75w"
Nov 04 03:43:22.264397 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:22.264366    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2c87d0f1-f6pvr"]
Nov 04 03:43:22.269871 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:43:22.269833    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc5519c5_c7a9_40e0_8eff_b4d0b8887f32.slice/crio-78edc16a9ddf4c3e4b72a437406d23e8534a56594c661c92bef4549923b60630 WatchSource:0}: Error finding container 78edc16a9ddf4c3e4b72a437406d23e8534a56594c661c92bef4549923b60630: Status 404 returned error can't find the container with id 78edc16a9ddf4c3e4b72a437406d23e8534a56594c661c92bef4549923b60630
Nov 04 03:43:23.086331 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:23.086283    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2c87d0f1-f6pvr" event={"ID":"fc5519c5-c7a9-40e0-8eff-b4d0b8887f32","Type":"ContainerStarted","Data":"5f63a717e08e6bcb32b14bee625d7388c48b47fe6d2edb88af0dfe9b0e465db6"}
Nov 04 03:43:23.087025 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:23.086986    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2c87d0f1-f6pvr" event={"ID":"fc5519c5-c7a9-40e0-8eff-b4d0b8887f32","Type":"ContainerStarted","Data":"78edc16a9ddf4c3e4b72a437406d23e8534a56594c661c92bef4549923b60630"}
Nov 04 03:43:23.203254 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:23.203205    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2c87d0f1-f6pvr" podStartSLOduration=2.203191372 podStartE2EDuration="2.203191372s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:43:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:43:23.202448683 +0000 UTC m=+1545.779101594" watchObservedRunningTime="2026-11-04 03:43:23.203191372 +0000 UTC m=+1545.779844309"
Nov 04 03:43:24.094673 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:24.094640    5609 generic.go:356] "Generic (PLEG): container finished" podID="fc5519c5-c7a9-40e0-8eff-b4d0b8887f32" containerID="5f63a717e08e6bcb32b14bee625d7388c48b47fe6d2edb88af0dfe9b0e465db6" exitCode=0
Nov 04 03:43:24.095067 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:24.094745    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2c87d0f1-f6pvr" event={"ID":"fc5519c5-c7a9-40e0-8eff-b4d0b8887f32","Type":"ContainerDied","Data":"5f63a717e08e6bcb32b14bee625d7388c48b47fe6d2edb88af0dfe9b0e465db6"}
Nov 04 03:43:25.238562 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:25.238537    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2c87d0f1-f6pvr"
Nov 04 03:43:25.358707 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:25.358622    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fc5519c5-c7a9-40e0-8eff-b4d0b8887f32-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fc5519c5-c7a9-40e0-8eff-b4d0b8887f32-maas-api-openshift-ca\") pod \"fc5519c5-c7a9-40e0-8eff-b4d0b8887f32\" (UID: \"fc5519c5-c7a9-40e0-8eff-b4d0b8887f32\") "
Nov 04 03:43:25.358866 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:25.358721    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fc5519c5-c7a9-40e0-8eff-b4d0b8887f32-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fc5519c5-c7a9-40e0-8eff-b4d0b8887f32-maas-api-xks-ca\") pod \"fc5519c5-c7a9-40e0-8eff-b4d0b8887f32\" (UID: \"fc5519c5-c7a9-40e0-8eff-b4d0b8887f32\") "
Nov 04 03:43:25.358952 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:25.358931    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fc5519c5-c7a9-40e0-8eff-b4d0b8887f32-kube-api-access-h86dk\" (UniqueName: \"kubernetes.io/projected/fc5519c5-c7a9-40e0-8eff-b4d0b8887f32-kube-api-access-h86dk\") pod \"fc5519c5-c7a9-40e0-8eff-b4d0b8887f32\" (UID: \"fc5519c5-c7a9-40e0-8eff-b4d0b8887f32\") "
Nov 04 03:43:25.359006 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:25.358964    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc5519c5-c7a9-40e0-8eff-b4d0b8887f32-maas-api-openshift-ca" pod "fc5519c5-c7a9-40e0-8eff-b4d0b8887f32" (UID: "fc5519c5-c7a9-40e0-8eff-b4d0b8887f32"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:43:25.359319 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:25.359295    5609 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fc5519c5-c7a9-40e0-8eff-b4d0b8887f32-maas-api-openshift-ca\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:25.360986 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:25.360959    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc5519c5-c7a9-40e0-8eff-b4d0b8887f32-maas-api-xks-ca" pod "fc5519c5-c7a9-40e0-8eff-b4d0b8887f32" (UID: "fc5519c5-c7a9-40e0-8eff-b4d0b8887f32"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:43:25.361447 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:25.361424    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc5519c5-c7a9-40e0-8eff-b4d0b8887f32-kube-api-access-h86dk" pod "fc5519c5-c7a9-40e0-8eff-b4d0b8887f32" (UID: "fc5519c5-c7a9-40e0-8eff-b4d0b8887f32"). InnerVolumeSpecName "kube-api-access-h86dk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:43:25.460265 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:25.460230    5609 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fc5519c5-c7a9-40e0-8eff-b4d0b8887f32-maas-api-xks-ca\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:25.460265 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:25.460258    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h86dk\" (UniqueName: \"kubernetes.io/projected/fc5519c5-c7a9-40e0-8eff-b4d0b8887f32-kube-api-access-h86dk\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:26.109825 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:26.109794    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2c87d0f1-f6pvr"
Nov 04 03:43:26.109825 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:26.109807    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2c87d0f1-f6pvr" event={"ID":"fc5519c5-c7a9-40e0-8eff-b4d0b8887f32","Type":"ContainerDied","Data":"78edc16a9ddf4c3e4b72a437406d23e8534a56594c661c92bef4549923b60630"}
Nov 04 03:43:26.110083 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:26.109854    5609 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="78edc16a9ddf4c3e4b72a437406d23e8534a56594c661c92bef4549923b60630"
Nov 04 03:43:26.473498 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:26.473455    5609 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w3-82c724-6d4b6d5bb8-8cdsz" podUID="d29f5b1d-29ba-4f10-b17d-d2c8ccd4cd0d" containerName="payload-processing" containerID="cri-o://80a4001a89dcc6ad0ebc1b898028f90abc3944e22b3f8c7f151b83b47b2a6c50" gracePeriod=25
Nov 04 03:43:26.588832 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:26.588794    5609 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-82c724-76bcd64bff-94mnv" podUID="d8178561-e4cd-488a-a88b-c62dfa4fee20" containerName="payload-pre-processing" containerID="cri-o://a5eec47bf85e1624e0ff01006d6065d67e0b0bab7fa3a686ed85e2936f6cd691" gracePeriod=25
Nov 04 03:43:26.737539 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:26.737478    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-82c724-6d4b6d5bb8-8cdsz"
Nov 04 03:43:26.887276 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:26.887241    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d29f5b1d-29ba-4f10-b17d-d2c8ccd4cd0d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d29f5b1d-29ba-4f10-b17d-d2c8ccd4cd0d-plugins-config-volume\") pod \"d29f5b1d-29ba-4f10-b17d-d2c8ccd4cd0d\" (UID: \"d29f5b1d-29ba-4f10-b17d-d2c8ccd4cd0d\") "
Nov 04 03:43:26.887468 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:26.887363    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d29f5b1d-29ba-4f10-b17d-d2c8ccd4cd0d-kube-api-access-27wzl\" (UniqueName: \"kubernetes.io/projected/d29f5b1d-29ba-4f10-b17d-d2c8ccd4cd0d-kube-api-access-27wzl\") pod \"d29f5b1d-29ba-4f10-b17d-d2c8ccd4cd0d\" (UID: \"d29f5b1d-29ba-4f10-b17d-d2c8ccd4cd0d\") "
Nov 04 03:43:26.887733 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:26.887705    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d29f5b1d-29ba-4f10-b17d-d2c8ccd4cd0d-plugins-config-volume" pod "d29f5b1d-29ba-4f10-b17d-d2c8ccd4cd0d" (UID: "d29f5b1d-29ba-4f10-b17d-d2c8ccd4cd0d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:43:26.889595 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:26.889560    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d29f5b1d-29ba-4f10-b17d-d2c8ccd4cd0d-kube-api-access-27wzl" pod "d29f5b1d-29ba-4f10-b17d-d2c8ccd4cd0d" (UID: "d29f5b1d-29ba-4f10-b17d-d2c8ccd4cd0d"). InnerVolumeSpecName "kube-api-access-27wzl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:43:26.988651 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:26.988562    5609 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d29f5b1d-29ba-4f10-b17d-d2c8ccd4cd0d-plugins-config-volume\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:26.988651 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:26.988592    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27wzl\" (UniqueName: \"kubernetes.io/projected/d29f5b1d-29ba-4f10-b17d-d2c8ccd4cd0d-kube-api-access-27wzl\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:27.117861 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:27.117822    5609 generic.go:356] "Generic (PLEG): container finished" podID="d29f5b1d-29ba-4f10-b17d-d2c8ccd4cd0d" containerID="80a4001a89dcc6ad0ebc1b898028f90abc3944e22b3f8c7f151b83b47b2a6c50" exitCode=0
Nov 04 03:43:27.118055 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:27.117867    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-82c724-6d4b6d5bb8-8cdsz"
Nov 04 03:43:27.118055 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:27.117902    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-82c724-6d4b6d5bb8-8cdsz" event={"ID":"d29f5b1d-29ba-4f10-b17d-d2c8ccd4cd0d","Type":"ContainerDied","Data":"80a4001a89dcc6ad0ebc1b898028f90abc3944e22b3f8c7f151b83b47b2a6c50"}
Nov 04 03:43:27.118055 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:27.117957    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-82c724-6d4b6d5bb8-8cdsz" event={"ID":"d29f5b1d-29ba-4f10-b17d-d2c8ccd4cd0d","Type":"ContainerDied","Data":"426b6289c8c968574e36762d4f871486854286efdcccdbd3d16abceb266ddc77"}
Nov 04 03:43:27.118055 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:27.117980    5609 scope.go:122] "RemoveContainer" containerID="80a4001a89dcc6ad0ebc1b898028f90abc3944e22b3f8c7f151b83b47b2a6c50"
Nov 04 03:43:27.129338 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:27.129316    5609 scope.go:122] "RemoveContainer" containerID="80a4001a89dcc6ad0ebc1b898028f90abc3944e22b3f8c7f151b83b47b2a6c50"
Nov 04 03:43:27.129721 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:43:27.129703    5609 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80a4001a89dcc6ad0ebc1b898028f90abc3944e22b3f8c7f151b83b47b2a6c50\": container with ID starting with 80a4001a89dcc6ad0ebc1b898028f90abc3944e22b3f8c7f151b83b47b2a6c50 not found: ID does not exist" containerID="80a4001a89dcc6ad0ebc1b898028f90abc3944e22b3f8c7f151b83b47b2a6c50"
Nov 04 03:43:27.129783 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:27.129729    5609 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80a4001a89dcc6ad0ebc1b898028f90abc3944e22b3f8c7f151b83b47b2a6c50"} err="failed to get container status \"80a4001a89dcc6ad0ebc1b898028f90abc3944e22b3f8c7f151b83b47b2a6c50\": rpc error: code = NotFound desc = could not find container \"80a4001a89dcc6ad0ebc1b898028f90abc3944e22b3f8c7f151b83b47b2a6c50\": container with ID starting with 80a4001a89dcc6ad0ebc1b898028f90abc3944e22b3f8c7f151b83b47b2a6c50 not found: ID does not exist"
Nov 04 03:43:27.275923 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:27.275827    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-82c724-6d4b6d5bb8-8cdsz"]
Nov 04 03:43:27.282574 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:27.282536    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-82c724-6d4b6d5bb8-8cdsz"]
Nov 04 03:43:27.535002 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:27.534912    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d29f5b1d-29ba-4f10-b17d-d2c8ccd4cd0d" path="/var/lib/kubelet/pods/d29f5b1d-29ba-4f10-b17d-d2c8ccd4cd0d/volumes"
Nov 04 03:43:32.033244 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:32.033206    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87"]
Nov 04 03:43:32.033992 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:32.033967    5609 cpu_manager.go:414] "containerMap: removing container" podUID="fc5519c5-c7a9-40e0-8eff-b4d0b8887f32" containerName="revoke-keys"
Nov 04 03:43:32.034033 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:32.033997    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fc5519c5-c7a9-40e0-8eff-b4d0b8887f32" containerName="revoke-keys"
Nov 04 03:43:32.034033 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:32.034022    5609 cpu_manager.go:414] "containerMap: removing container" podUID="d29f5b1d-29ba-4f10-b17d-d2c8ccd4cd0d" containerName="payload-processing"
Nov 04 03:43:32.034093 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:32.034030    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d29f5b1d-29ba-4f10-b17d-d2c8ccd4cd0d" containerName="payload-processing"
Nov 04 03:43:32.034230 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:32.034216    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="fc5519c5-c7a9-40e0-8eff-b4d0b8887f32" containerName="revoke-keys"
Nov 04 03:43:32.034261 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:32.034236    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="d29f5b1d-29ba-4f10-b17d-d2c8ccd4cd0d" containerName="payload-processing"
Nov 04 03:43:32.039666 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:32.039639    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87"
Nov 04 03:43:32.114685 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:32.114659    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-50429e-openshift-default-dockercfg-96qjd\""
Nov 04 03:43:32.184161 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:32.167214    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87"]
Nov 04 03:43:32.184442 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:32.169008    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72bfd449-2341-4be8-8e05-4df3fd79b8b6-istio-envoy\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87\" (UID: \"72bfd449-2341-4be8-8e05-4df3fd79b8b6\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87"
Nov 04 03:43:32.184768 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:32.184747    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8n8b\" (UniqueName: \"kubernetes.io/projected/72bfd449-2341-4be8-8e05-4df3fd79b8b6-kube-api-access-v8n8b\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87\" (UID: \"72bfd449-2341-4be8-8e05-4df3fd79b8b6\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87"
Nov 04 03:43:32.184941 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:32.184913    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72bfd449-2341-4be8-8e05-4df3fd79b8b6-istio-podinfo\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87\" (UID: \"72bfd449-2341-4be8-8e05-4df3fd79b8b6\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87"
Nov 04 03:43:32.185058 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:32.185045    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72bfd449-2341-4be8-8e05-4df3fd79b8b6-workload-certs\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87\" (UID: \"72bfd449-2341-4be8-8e05-4df3fd79b8b6\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87"
Nov 04 03:43:32.185236 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:32.185224    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72bfd449-2341-4be8-8e05-4df3fd79b8b6-istiod-ca-cert\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87\" (UID: \"72bfd449-2341-4be8-8e05-4df3fd79b8b6\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87"
Nov 04 03:43:32.185419 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:32.185393    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72bfd449-2341-4be8-8e05-4df3fd79b8b6-credential-socket\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87\" (UID: \"72bfd449-2341-4be8-8e05-4df3fd79b8b6\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87"
Nov 04 03:43:32.185561 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:32.185549    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72bfd449-2341-4be8-8e05-4df3fd79b8b6-istio-data\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87\" (UID: \"72bfd449-2341-4be8-8e05-4df3fd79b8b6\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87"
Nov 04 03:43:32.185675 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:32.185663    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72bfd449-2341-4be8-8e05-4df3fd79b8b6-istio-token\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87\" (UID: \"72bfd449-2341-4be8-8e05-4df3fd79b8b6\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87"
Nov 04 03:43:32.185823 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:32.185799    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72bfd449-2341-4be8-8e05-4df3fd79b8b6-workload-socket\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87\" (UID: \"72bfd449-2341-4be8-8e05-4df3fd79b8b6\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87"
Nov 04 03:43:32.288176 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:32.288077    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72bfd449-2341-4be8-8e05-4df3fd79b8b6-istiod-ca-cert\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87\" (UID: \"72bfd449-2341-4be8-8e05-4df3fd79b8b6\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87"
Nov 04 03:43:32.294489 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:32.294447    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72bfd449-2341-4be8-8e05-4df3fd79b8b6-credential-socket\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87\" (UID: \"72bfd449-2341-4be8-8e05-4df3fd79b8b6\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87"
Nov 04 03:43:32.294654 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:32.294520    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72bfd449-2341-4be8-8e05-4df3fd79b8b6-istio-data\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87\" (UID: \"72bfd449-2341-4be8-8e05-4df3fd79b8b6\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87"
Nov 04 03:43:32.294654 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:32.294560    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72bfd449-2341-4be8-8e05-4df3fd79b8b6-istio-token\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87\" (UID: \"72bfd449-2341-4be8-8e05-4df3fd79b8b6\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87"
Nov 04 03:43:32.294654 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:32.294608    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72bfd449-2341-4be8-8e05-4df3fd79b8b6-workload-socket\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87\" (UID: \"72bfd449-2341-4be8-8e05-4df3fd79b8b6\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87"
Nov 04 03:43:32.294830 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:32.294684    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72bfd449-2341-4be8-8e05-4df3fd79b8b6-istio-envoy\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87\" (UID: \"72bfd449-2341-4be8-8e05-4df3fd79b8b6\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87"
Nov 04 03:43:32.294830 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:32.294731    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8n8b\" (UniqueName: \"kubernetes.io/projected/72bfd449-2341-4be8-8e05-4df3fd79b8b6-kube-api-access-v8n8b\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87\" (UID: \"72bfd449-2341-4be8-8e05-4df3fd79b8b6\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87"
Nov 04 03:43:32.294830 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:32.294786    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72bfd449-2341-4be8-8e05-4df3fd79b8b6-istio-podinfo\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87\" (UID: \"72bfd449-2341-4be8-8e05-4df3fd79b8b6\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87"
Nov 04 03:43:32.294830 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:32.294813    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72bfd449-2341-4be8-8e05-4df3fd79b8b6-workload-certs\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87\" (UID: \"72bfd449-2341-4be8-8e05-4df3fd79b8b6\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87"
Nov 04 03:43:32.295360 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:32.295330    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72bfd449-2341-4be8-8e05-4df3fd79b8b6-workload-certs\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87\" (UID: \"72bfd449-2341-4be8-8e05-4df3fd79b8b6\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87"
Nov 04 03:43:32.301351 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:32.301325    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72bfd449-2341-4be8-8e05-4df3fd79b8b6-istio-data\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87\" (UID: \"72bfd449-2341-4be8-8e05-4df3fd79b8b6\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87"
Nov 04 03:43:32.301822 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:32.301765    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72bfd449-2341-4be8-8e05-4df3fd79b8b6-credential-socket\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87\" (UID: \"72bfd449-2341-4be8-8e05-4df3fd79b8b6\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87"
Nov 04 03:43:32.304425 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:32.301957    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72bfd449-2341-4be8-8e05-4df3fd79b8b6-workload-socket\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87\" (UID: \"72bfd449-2341-4be8-8e05-4df3fd79b8b6\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87"
Nov 04 03:43:32.304939 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:32.304914    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72bfd449-2341-4be8-8e05-4df3fd79b8b6-istiod-ca-cert\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87\" (UID: \"72bfd449-2341-4be8-8e05-4df3fd79b8b6\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87"
Nov 04 03:43:32.315432 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:32.310768    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72bfd449-2341-4be8-8e05-4df3fd79b8b6-istio-podinfo\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87\" (UID: \"72bfd449-2341-4be8-8e05-4df3fd79b8b6\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87"
Nov 04 03:43:32.333934 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:32.333690    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72bfd449-2341-4be8-8e05-4df3fd79b8b6-istio-envoy\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87\" (UID: \"72bfd449-2341-4be8-8e05-4df3fd79b8b6\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87"
Nov 04 03:43:32.365060 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:32.365028    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72bfd449-2341-4be8-8e05-4df3fd79b8b6-istio-token\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87\" (UID: \"72bfd449-2341-4be8-8e05-4df3fd79b8b6\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87"
Nov 04 03:43:32.371168 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:32.371126    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8n8b\" (UniqueName: \"kubernetes.io/projected/72bfd449-2341-4be8-8e05-4df3fd79b8b6-kube-api-access-v8n8b\") pod \"e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87\" (UID: \"72bfd449-2341-4be8-8e05-4df3fd79b8b6\") " pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87"
Nov 04 03:43:32.374542 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:32.374514    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87"
Nov 04 03:43:32.913264 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:32.913234    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87"]
Nov 04 03:43:32.915549 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:43:32.915521    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72bfd449_2341_4be8_8e05_4df3fd79b8b6.slice/crio-e30f1add728d1c8293fbbcc05fe116fbf9a22b00476aa9bde7df28d7df93f2d5 WatchSource:0}: Error finding container e30f1add728d1c8293fbbcc05fe116fbf9a22b00476aa9bde7df28d7df93f2d5: Status 404 returned error can't find the container with id e30f1add728d1c8293fbbcc05fe116fbf9a22b00476aa9bde7df28d7df93f2d5
Nov 04 03:43:32.921302 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:32.918607    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:43:32.921302 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:32.918694    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:43:32.921302 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:32.918738    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:43:33.207425 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:33.204831    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87" event={"ID":"72bfd449-2341-4be8-8e05-4df3fd79b8b6","Type":"ContainerStarted","Data":"fac1d84e0004d481c7849ffde578c586e702418aef86c557820a1c4987b64cd0"}
Nov 04 03:43:33.207425 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:33.204903    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87" event={"ID":"72bfd449-2341-4be8-8e05-4df3fd79b8b6","Type":"ContainerStarted","Data":"e30f1add728d1c8293fbbcc05fe116fbf9a22b00476aa9bde7df28d7df93f2d5"}
Nov 04 03:43:33.332006 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:33.331940    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87" podStartSLOduration=2.331920677 podStartE2EDuration="2.331920677s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:43:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:43:33.330799097 +0000 UTC m=+1555.907452011" watchObservedRunningTime="2026-11-04 03:43:33.331920677 +0000 UTC m=+1555.908573606"
Nov 04 03:43:33.375676 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:33.375627    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87"
Nov 04 03:43:33.377883 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:33.377851    5609 patch_prober.go:29] interesting pod/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Nov 04 03:43:33.378414 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:33.378083    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87" podUID="72bfd449-2341-4be8-8e05-4df3fd79b8b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Nov 04 03:43:34.378424 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:34.377799    5609 patch_prober.go:29] interesting pod/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Nov 04 03:43:34.378424 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:34.377858    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87" podUID="72bfd449-2341-4be8-8e05-4df3fd79b8b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Nov 04 03:43:35.378426 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:35.376811    5609 patch_prober.go:29] interesting pod/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Nov 04 03:43:35.378426 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:35.376873    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87" podUID="72bfd449-2341-4be8-8e05-4df3fd79b8b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Nov 04 03:43:36.129595 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:36.129555    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9ec8eb24-glglh"]
Nov 04 03:43:36.133523 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:36.133500    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ec8eb24-glglh"
Nov 04 03:43:36.196422 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:36.193052    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9ec8eb24-glglh\" (UID: \"3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ec8eb24-glglh"
Nov 04 03:43:36.196422 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:36.193112    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9ec8eb24-glglh\" (UID: \"3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ec8eb24-glglh"
Nov 04 03:43:36.196422 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:36.193194    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2b88k\" (UniqueName: \"kubernetes.io/projected/3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3-kube-api-access-2b88k\") pod \"maas-api-revoke-sub-9ec8eb24-glglh\" (UID: \"3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ec8eb24-glglh"
Nov 04 03:43:36.196422 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:36.194196    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 03:43:36.196422 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:36.194312    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x9cnx\""
Nov 04 03:43:36.246071 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:36.246035    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9ec8eb24-glglh"]
Nov 04 03:43:36.308024 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:36.307982    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2b88k\" (UniqueName: \"kubernetes.io/projected/3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3-kube-api-access-2b88k\") pod \"maas-api-revoke-sub-9ec8eb24-glglh\" (UID: \"3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ec8eb24-glglh"
Nov 04 03:43:36.308820 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:36.308792    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9ec8eb24-glglh\" (UID: \"3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ec8eb24-glglh"
Nov 04 03:43:36.309788 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:36.309735    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9ec8eb24-glglh\" (UID: \"3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ec8eb24-glglh"
Nov 04 03:43:36.313924 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:36.309679    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9ec8eb24-glglh\" (UID: \"3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ec8eb24-glglh"
Nov 04 03:43:36.321785 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:36.321756    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9ec8eb24-glglh\" (UID: \"3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ec8eb24-glglh"
Nov 04 03:43:36.366833 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:36.366806    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2b88k\" (UniqueName: \"kubernetes.io/projected/3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3-kube-api-access-2b88k\") pod \"maas-api-revoke-sub-9ec8eb24-glglh\" (UID: \"3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ec8eb24-glglh"
Nov 04 03:43:36.375323 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:36.375296    5609 patch_prober.go:29] interesting pod/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Nov 04 03:43:36.375463 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:36.375340    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87" podUID="72bfd449-2341-4be8-8e05-4df3fd79b8b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Nov 04 03:43:36.414851 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:36.414760    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-82c724-76bcd64bff-94mnv" podUID="d8178561-e4cd-488a-a88b-c62dfa4fee20" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:43:36.447542 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:36.447513    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ec8eb24-glglh"
Nov 04 03:43:36.866392 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:36.866355    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9ec8eb24-glglh"]
Nov 04 03:43:36.870920 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:43:36.870887    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a0d0bd8_f6ea_4db3_9c80_6ea67688d0e3.slice/crio-1025210a083d0bec407993a9f38305b3d1b1dcd234698694fc7102dfdff62048 WatchSource:0}: Error finding container 1025210a083d0bec407993a9f38305b3d1b1dcd234698694fc7102dfdff62048: Status 404 returned error can't find the container with id 1025210a083d0bec407993a9f38305b3d1b1dcd234698694fc7102dfdff62048
Nov 04 03:43:37.278696 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:37.278662    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ec8eb24-glglh" event={"ID":"3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3","Type":"ContainerStarted","Data":"eb36c275241083a786c84d8d16a3d22ee7fb8d6ef6df36e4a22a0e4ceca1e888"}
Nov 04 03:43:37.278842 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:37.278716    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ec8eb24-glglh" event={"ID":"3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3","Type":"ContainerStarted","Data":"1025210a083d0bec407993a9f38305b3d1b1dcd234698694fc7102dfdff62048"}
Nov 04 03:43:37.375218 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:37.375183    5609 patch_prober.go:29] interesting pod/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Nov 04 03:43:37.375360 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:37.375231    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87" podUID="72bfd449-2341-4be8-8e05-4df3fd79b8b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Nov 04 03:43:37.394039 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:37.393983    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ec8eb24-glglh" podStartSLOduration=1.393963673 podStartE2EDuration="1.393963673s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:43:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:43:37.391872987 +0000 UTC m=+1559.968525901" watchObservedRunningTime="2026-11-04 03:43:37.393963673 +0000 UTC m=+1559.970616590"
Nov 04 03:43:38.302481 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:38.292287    5609 generic.go:356] "Generic (PLEG): container finished" podID="3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3" containerID="eb36c275241083a786c84d8d16a3d22ee7fb8d6ef6df36e4a22a0e4ceca1e888" exitCode=0
Nov 04 03:43:38.302481 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:38.292477    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ec8eb24-glglh" event={"ID":"3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3","Type":"ContainerDied","Data":"eb36c275241083a786c84d8d16a3d22ee7fb8d6ef6df36e4a22a0e4ceca1e888"}
Nov 04 03:43:38.375000 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:38.374964    5609 patch_prober.go:29] interesting pod/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Nov 04 03:43:38.375170 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:38.375013    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87" podUID="72bfd449-2341-4be8-8e05-4df3fd79b8b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Nov 04 03:43:39.380683 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:39.380385    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87"
Nov 04 03:43:39.381660 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:39.381614    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87"
Nov 04 03:43:39.382673 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:39.382650    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87"
Nov 04 03:43:39.447771 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:39.447736    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ec8eb24-glglh"
Nov 04 03:43:39.573654 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:39.573574    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3-maas-api-xks-ca\") pod \"3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3\" (UID: \"3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3\") "
Nov 04 03:43:39.573654 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:39.573629    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3-maas-api-openshift-ca\") pod \"3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3\" (UID: \"3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3\") "
Nov 04 03:43:39.573847 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:39.573672    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3-kube-api-access-2b88k\" (UniqueName: \"kubernetes.io/projected/3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3-kube-api-access-2b88k\") pod \"3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3\" (UID: \"3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3\") "
Nov 04 03:43:39.574114 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:39.574072    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3-maas-api-openshift-ca" pod "3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3" (UID: "3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:43:39.575766 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:39.575735    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3-maas-api-xks-ca" pod "3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3" (UID: "3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:43:39.575870 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:39.575833    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3-kube-api-access-2b88k" pod "3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3" (UID: "3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3"). InnerVolumeSpecName "kube-api-access-2b88k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:43:39.676670 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:39.676267    5609 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3-maas-api-xks-ca\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:39.676670 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:39.676302    5609 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3-maas-api-openshift-ca\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:39.676670 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:39.676319    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2b88k\" (UniqueName: \"kubernetes.io/projected/3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3-kube-api-access-2b88k\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:40.309362 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:40.309325    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ec8eb24-glglh"
Nov 04 03:43:40.309569 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:40.309357    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ec8eb24-glglh" event={"ID":"3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3","Type":"ContainerDied","Data":"1025210a083d0bec407993a9f38305b3d1b1dcd234698694fc7102dfdff62048"}
Nov 04 03:43:40.309569 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:40.309438    5609 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1025210a083d0bec407993a9f38305b3d1b1dcd234698694fc7102dfdff62048"
Nov 04 03:43:46.409215 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:46.409175    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-82c724-76bcd64bff-94mnv" podUID="d8178561-e4cd-488a-a88b-c62dfa4fee20" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:43:46.579776 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:46.579741    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87"]
Nov 04 03:43:46.579984 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:46.579958    5609 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87" podUID="72bfd449-2341-4be8-8e05-4df3fd79b8b6" containerName="istio-proxy" containerID="cri-o://fac1d84e0004d481c7849ffde578c586e702418aef86c557820a1c4987b64cd0" gracePeriod=30
Nov 04 03:43:46.899048 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:46.899015    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-82c724-76bcd64bff-94mnv"
Nov 04 03:43:47.014393 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:47.014316    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d8178561-e4cd-488a-a88b-c62dfa4fee20-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8178561-e4cd-488a-a88b-c62dfa4fee20-plugins-config-volume\") pod \"d8178561-e4cd-488a-a88b-c62dfa4fee20\" (UID: \"d8178561-e4cd-488a-a88b-c62dfa4fee20\") "
Nov 04 03:43:47.014598 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:47.014501    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d8178561-e4cd-488a-a88b-c62dfa4fee20-kube-api-access-4xkb4\" (UniqueName: \"kubernetes.io/projected/d8178561-e4cd-488a-a88b-c62dfa4fee20-kube-api-access-4xkb4\") pod \"d8178561-e4cd-488a-a88b-c62dfa4fee20\" (UID: \"d8178561-e4cd-488a-a88b-c62dfa4fee20\") "
Nov 04 03:43:47.014766 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:47.014744    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8178561-e4cd-488a-a88b-c62dfa4fee20-plugins-config-volume" pod "d8178561-e4cd-488a-a88b-c62dfa4fee20" (UID: "d8178561-e4cd-488a-a88b-c62dfa4fee20"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:43:47.014959 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:47.014928    5609 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8178561-e4cd-488a-a88b-c62dfa4fee20-plugins-config-volume\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:47.017003 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:47.016973    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8178561-e4cd-488a-a88b-c62dfa4fee20-kube-api-access-4xkb4" pod "d8178561-e4cd-488a-a88b-c62dfa4fee20" (UID: "d8178561-e4cd-488a-a88b-c62dfa4fee20"). InnerVolumeSpecName "kube-api-access-4xkb4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:43:47.116436 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:47.116389    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4xkb4\" (UniqueName: \"kubernetes.io/projected/d8178561-e4cd-488a-a88b-c62dfa4fee20-kube-api-access-4xkb4\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:47.357038 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:47.356943    5609 generic.go:356] "Generic (PLEG): container finished" podID="d8178561-e4cd-488a-a88b-c62dfa4fee20" containerID="a5eec47bf85e1624e0ff01006d6065d67e0b0bab7fa3a686ed85e2936f6cd691" exitCode=0
Nov 04 03:43:47.357038 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:47.356997    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-82c724-76bcd64bff-94mnv"
Nov 04 03:43:47.357038 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:47.357030    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-82c724-76bcd64bff-94mnv" event={"ID":"d8178561-e4cd-488a-a88b-c62dfa4fee20","Type":"ContainerDied","Data":"a5eec47bf85e1624e0ff01006d6065d67e0b0bab7fa3a686ed85e2936f6cd691"}
Nov 04 03:43:47.357325 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:47.357096    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-82c724-76bcd64bff-94mnv" event={"ID":"d8178561-e4cd-488a-a88b-c62dfa4fee20","Type":"ContainerDied","Data":"be72a982ca52618b86ca64b206796c64721c36883ee90e9c796278111ddb38b2"}
Nov 04 03:43:47.357325 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:47.357119    5609 scope.go:122] "RemoveContainer" containerID="a5eec47bf85e1624e0ff01006d6065d67e0b0bab7fa3a686ed85e2936f6cd691"
Nov 04 03:43:47.368310 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:47.368291    5609 scope.go:122] "RemoveContainer" containerID="a5eec47bf85e1624e0ff01006d6065d67e0b0bab7fa3a686ed85e2936f6cd691"
Nov 04 03:43:47.368633 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:43:47.368605    5609 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5eec47bf85e1624e0ff01006d6065d67e0b0bab7fa3a686ed85e2936f6cd691\": container with ID starting with a5eec47bf85e1624e0ff01006d6065d67e0b0bab7fa3a686ed85e2936f6cd691 not found: ID does not exist" containerID="a5eec47bf85e1624e0ff01006d6065d67e0b0bab7fa3a686ed85e2936f6cd691"
Nov 04 03:43:47.368720 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:47.368645    5609 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5eec47bf85e1624e0ff01006d6065d67e0b0bab7fa3a686ed85e2936f6cd691"} err="failed to get container status \"a5eec47bf85e1624e0ff01006d6065d67e0b0bab7fa3a686ed85e2936f6cd691\": rpc error: code = NotFound desc = could not find container \"a5eec47bf85e1624e0ff01006d6065d67e0b0bab7fa3a686ed85e2936f6cd691\": container with ID starting with a5eec47bf85e1624e0ff01006d6065d67e0b0bab7fa3a686ed85e2936f6cd691 not found: ID does not exist"
Nov 04 03:43:47.514759 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:47.514724    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-82c724-76bcd64bff-94mnv"]
Nov 04 03:43:47.519148 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:47.519117    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-82c724-76bcd64bff-94mnv"]
Nov 04 03:43:47.534663 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:47.534635    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d8178561-e4cd-488a-a88b-c62dfa4fee20" path="/var/lib/kubelet/pods/d8178561-e4cd-488a-a88b-c62dfa4fee20/volumes"
Nov 04 03:43:48.561063 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:43:48.561026    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:43:49.830309 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:49.830268    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b"]
Nov 04 03:43:49.830739 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:49.830546    5609 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b" podUID="3e2b497b-98dc-4fa4-bbb5-a357f795628d" containerName="istio-proxy" containerID="cri-o://52113e6d5858fda7d97ed01e115b705fc8dcad081c482d8bfdcb0129e97fd285" gracePeriod=30
Nov 04 03:43:51.848755 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:51.848695    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87"
Nov 04 03:43:51.973278 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:51.973246    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/72bfd449-2341-4be8-8e05-4df3fd79b8b6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72bfd449-2341-4be8-8e05-4df3fd79b8b6-credential-socket\") pod \"72bfd449-2341-4be8-8e05-4df3fd79b8b6\" (UID: \"72bfd449-2341-4be8-8e05-4df3fd79b8b6\") "
Nov 04 03:43:51.973474 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:51.973293    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/72bfd449-2341-4be8-8e05-4df3fd79b8b6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72bfd449-2341-4be8-8e05-4df3fd79b8b6-istiod-ca-cert\") pod \"72bfd449-2341-4be8-8e05-4df3fd79b8b6\" (UID: \"72bfd449-2341-4be8-8e05-4df3fd79b8b6\") "
Nov 04 03:43:51.973474 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:51.973333    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/72bfd449-2341-4be8-8e05-4df3fd79b8b6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72bfd449-2341-4be8-8e05-4df3fd79b8b6-workload-certs\") pod \"72bfd449-2341-4be8-8e05-4df3fd79b8b6\" (UID: \"72bfd449-2341-4be8-8e05-4df3fd79b8b6\") "
Nov 04 03:43:51.973474 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:51.973356    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/72bfd449-2341-4be8-8e05-4df3fd79b8b6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72bfd449-2341-4be8-8e05-4df3fd79b8b6-istio-envoy\") pod \"72bfd449-2341-4be8-8e05-4df3fd79b8b6\" (UID: \"72bfd449-2341-4be8-8e05-4df3fd79b8b6\") "
Nov 04 03:43:51.973474 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:51.973420    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/72bfd449-2341-4be8-8e05-4df3fd79b8b6-kube-api-access-v8n8b\" (UniqueName: \"kubernetes.io/projected/72bfd449-2341-4be8-8e05-4df3fd79b8b6-kube-api-access-v8n8b\") pod \"72bfd449-2341-4be8-8e05-4df3fd79b8b6\" (UID: \"72bfd449-2341-4be8-8e05-4df3fd79b8b6\") "
Nov 04 03:43:51.973474 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:51.973447    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/72bfd449-2341-4be8-8e05-4df3fd79b8b6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72bfd449-2341-4be8-8e05-4df3fd79b8b6-istio-data\") pod \"72bfd449-2341-4be8-8e05-4df3fd79b8b6\" (UID: \"72bfd449-2341-4be8-8e05-4df3fd79b8b6\") "
Nov 04 03:43:51.973747 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:51.973555    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/72bfd449-2341-4be8-8e05-4df3fd79b8b6-istio-token\" (UniqueName: \"kubernetes.io/projected/72bfd449-2341-4be8-8e05-4df3fd79b8b6-istio-token\") pod \"72bfd449-2341-4be8-8e05-4df3fd79b8b6\" (UID: \"72bfd449-2341-4be8-8e05-4df3fd79b8b6\") "
Nov 04 03:43:51.973747 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:51.973591    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72bfd449-2341-4be8-8e05-4df3fd79b8b6-credential-socket" pod "72bfd449-2341-4be8-8e05-4df3fd79b8b6" (UID: "72bfd449-2341-4be8-8e05-4df3fd79b8b6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:43:51.973747 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:51.973638    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72bfd449-2341-4be8-8e05-4df3fd79b8b6-workload-certs" pod "72bfd449-2341-4be8-8e05-4df3fd79b8b6" (UID: "72bfd449-2341-4be8-8e05-4df3fd79b8b6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:43:51.973747 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:51.973672    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/72bfd449-2341-4be8-8e05-4df3fd79b8b6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72bfd449-2341-4be8-8e05-4df3fd79b8b6-workload-socket\") pod \"72bfd449-2341-4be8-8e05-4df3fd79b8b6\" (UID: \"72bfd449-2341-4be8-8e05-4df3fd79b8b6\") "
Nov 04 03:43:51.973941 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:51.973745    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/72bfd449-2341-4be8-8e05-4df3fd79b8b6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72bfd449-2341-4be8-8e05-4df3fd79b8b6-istio-podinfo\") pod \"72bfd449-2341-4be8-8e05-4df3fd79b8b6\" (UID: \"72bfd449-2341-4be8-8e05-4df3fd79b8b6\") "
Nov 04 03:43:51.973941 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:51.973784    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72bfd449-2341-4be8-8e05-4df3fd79b8b6-istio-data" pod "72bfd449-2341-4be8-8e05-4df3fd79b8b6" (UID: "72bfd449-2341-4be8-8e05-4df3fd79b8b6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:43:51.973941 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:51.973907    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72bfd449-2341-4be8-8e05-4df3fd79b8b6-workload-socket" pod "72bfd449-2341-4be8-8e05-4df3fd79b8b6" (UID: "72bfd449-2341-4be8-8e05-4df3fd79b8b6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:43:51.974117 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:51.974094    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72bfd449-2341-4be8-8e05-4df3fd79b8b6-istiod-ca-cert" pod "72bfd449-2341-4be8-8e05-4df3fd79b8b6" (UID: "72bfd449-2341-4be8-8e05-4df3fd79b8b6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:43:51.974260 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:51.974243    5609 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72bfd449-2341-4be8-8e05-4df3fd79b8b6-workload-socket\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:51.974340 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:51.974266    5609 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72bfd449-2341-4be8-8e05-4df3fd79b8b6-credential-socket\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:51.974340 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:51.974279    5609 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72bfd449-2341-4be8-8e05-4df3fd79b8b6-istiod-ca-cert\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:51.974340 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:51.974292    5609 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72bfd449-2341-4be8-8e05-4df3fd79b8b6-workload-certs\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:51.974340 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:51.974304    5609 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72bfd449-2341-4be8-8e05-4df3fd79b8b6-istio-data\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:51.975802 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:51.975773    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72bfd449-2341-4be8-8e05-4df3fd79b8b6-kube-api-access-v8n8b" pod "72bfd449-2341-4be8-8e05-4df3fd79b8b6" (UID: "72bfd449-2341-4be8-8e05-4df3fd79b8b6"). InnerVolumeSpecName "kube-api-access-v8n8b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:43:51.975931 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:51.975833    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72bfd449-2341-4be8-8e05-4df3fd79b8b6-istio-token" pod "72bfd449-2341-4be8-8e05-4df3fd79b8b6" (UID: "72bfd449-2341-4be8-8e05-4df3fd79b8b6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:43:51.976166 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:51.976149    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72bfd449-2341-4be8-8e05-4df3fd79b8b6-istio-envoy" pod "72bfd449-2341-4be8-8e05-4df3fd79b8b6" (UID: "72bfd449-2341-4be8-8e05-4df3fd79b8b6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:43:51.976166 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:51.976154    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/72bfd449-2341-4be8-8e05-4df3fd79b8b6-istio-podinfo" pod "72bfd449-2341-4be8-8e05-4df3fd79b8b6" (UID: "72bfd449-2341-4be8-8e05-4df3fd79b8b6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:43:52.075927 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:52.075888    5609 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72bfd449-2341-4be8-8e05-4df3fd79b8b6-istio-envoy\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:52.075927 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:52.075917    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v8n8b\" (UniqueName: \"kubernetes.io/projected/72bfd449-2341-4be8-8e05-4df3fd79b8b6-kube-api-access-v8n8b\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:52.075927 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:52.075926    5609 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72bfd449-2341-4be8-8e05-4df3fd79b8b6-istio-token\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:52.075927 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:52.075934    5609 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72bfd449-2341-4be8-8e05-4df3fd79b8b6-istio-podinfo\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:52.391381 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:52.391344    5609 generic.go:356] "Generic (PLEG): container finished" podID="72bfd449-2341-4be8-8e05-4df3fd79b8b6" containerID="fac1d84e0004d481c7849ffde578c586e702418aef86c557820a1c4987b64cd0" exitCode=0
Nov 04 03:43:52.391594 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:52.391419    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87"
Nov 04 03:43:52.391594 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:52.391434    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87" event={"ID":"72bfd449-2341-4be8-8e05-4df3fd79b8b6","Type":"ContainerDied","Data":"fac1d84e0004d481c7849ffde578c586e702418aef86c557820a1c4987b64cd0"}
Nov 04 03:43:52.391594 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:52.391524    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87" event={"ID":"72bfd449-2341-4be8-8e05-4df3fd79b8b6","Type":"ContainerDied","Data":"e30f1add728d1c8293fbbcc05fe116fbf9a22b00476aa9bde7df28d7df93f2d5"}
Nov 04 03:43:52.391594 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:52.391554    5609 scope.go:122] "RemoveContainer" containerID="fac1d84e0004d481c7849ffde578c586e702418aef86c557820a1c4987b64cd0"
Nov 04 03:43:52.402514 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:52.402491    5609 scope.go:122] "RemoveContainer" containerID="fac1d84e0004d481c7849ffde578c586e702418aef86c557820a1c4987b64cd0"
Nov 04 03:43:52.402785 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:43:52.402766    5609 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fac1d84e0004d481c7849ffde578c586e702418aef86c557820a1c4987b64cd0\": container with ID starting with fac1d84e0004d481c7849ffde578c586e702418aef86c557820a1c4987b64cd0 not found: ID does not exist" containerID="fac1d84e0004d481c7849ffde578c586e702418aef86c557820a1c4987b64cd0"
Nov 04 03:43:52.402834 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:52.402794    5609 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fac1d84e0004d481c7849ffde578c586e702418aef86c557820a1c4987b64cd0"} err="failed to get container status \"fac1d84e0004d481c7849ffde578c586e702418aef86c557820a1c4987b64cd0\": rpc error: code = NotFound desc = could not find container \"fac1d84e0004d481c7849ffde578c586e702418aef86c557820a1c4987b64cd0\": container with ID starting with fac1d84e0004d481c7849ffde578c586e702418aef86c557820a1c4987b64cd0 not found: ID does not exist"
Nov 04 03:43:52.565503 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:52.564725    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87"]
Nov 04 03:43:52.568157 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:52.568129    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-50429e-openshift-default-755c7b9cf9-xsv87"]
Nov 04 03:43:53.536308 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:53.536273    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="72bfd449-2341-4be8-8e05-4df3fd79b8b6" path="/var/lib/kubelet/pods/72bfd449-2341-4be8-8e05-4df3fd79b8b6/volumes"
Nov 04 03:43:55.483151 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:55.483120    5609 generic.go:356] "Generic (PLEG): container finished" podID="3e2b497b-98dc-4fa4-bbb5-a357f795628d" containerID="52113e6d5858fda7d97ed01e115b705fc8dcad081c482d8bfdcb0129e97fd285" exitCode=0
Nov 04 03:43:55.483589 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:55.483246    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b" event={"ID":"3e2b497b-98dc-4fa4-bbb5-a357f795628d","Type":"ContainerDied","Data":"52113e6d5858fda7d97ed01e115b705fc8dcad081c482d8bfdcb0129e97fd285"}
Nov 04 03:43:55.483589 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:55.483328    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b" event={"ID":"3e2b497b-98dc-4fa4-bbb5-a357f795628d","Type":"ContainerDied","Data":"80b8c6ef0d8da20f8907c08a92552b7f1ed0c9402befded34071a1c4e4267f33"}
Nov 04 03:43:55.483589 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:55.483354    5609 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="80b8c6ef0d8da20f8907c08a92552b7f1ed0c9402befded34071a1c4e4267f33"
Nov 04 03:43:55.519440 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:55.514956    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b"
Nov 04 03:43:55.676664 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:55.676620    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3e2b497b-98dc-4fa4-bbb5-a357f795628d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e2b497b-98dc-4fa4-bbb5-a357f795628d-workload-socket\") pod \"3e2b497b-98dc-4fa4-bbb5-a357f795628d\" (UID: \"3e2b497b-98dc-4fa4-bbb5-a357f795628d\") "
Nov 04 03:43:55.676664 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:55.676666    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3e2b497b-98dc-4fa4-bbb5-a357f795628d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e2b497b-98dc-4fa4-bbb5-a357f795628d-credential-socket\") pod \"3e2b497b-98dc-4fa4-bbb5-a357f795628d\" (UID: \"3e2b497b-98dc-4fa4-bbb5-a357f795628d\") "
Nov 04 03:43:55.676927 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:55.676769    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3e2b497b-98dc-4fa4-bbb5-a357f795628d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e2b497b-98dc-4fa4-bbb5-a357f795628d-workload-certs\") pod \"3e2b497b-98dc-4fa4-bbb5-a357f795628d\" (UID: \"3e2b497b-98dc-4fa4-bbb5-a357f795628d\") "
Nov 04 03:43:55.676927 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:55.676842    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3e2b497b-98dc-4fa4-bbb5-a357f795628d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e2b497b-98dc-4fa4-bbb5-a357f795628d-istio-data\") pod \"3e2b497b-98dc-4fa4-bbb5-a357f795628d\" (UID: \"3e2b497b-98dc-4fa4-bbb5-a357f795628d\") "
Nov 04 03:43:55.676927 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:55.676882    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3e2b497b-98dc-4fa4-bbb5-a357f795628d-kube-api-access-9sj7t\" (UniqueName: \"kubernetes.io/projected/3e2b497b-98dc-4fa4-bbb5-a357f795628d-kube-api-access-9sj7t\") pod \"3e2b497b-98dc-4fa4-bbb5-a357f795628d\" (UID: \"3e2b497b-98dc-4fa4-bbb5-a357f795628d\") "
Nov 04 03:43:55.676927 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:55.676913    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3e2b497b-98dc-4fa4-bbb5-a357f795628d-istio-token\" (UniqueName: \"kubernetes.io/projected/3e2b497b-98dc-4fa4-bbb5-a357f795628d-istio-token\") pod \"3e2b497b-98dc-4fa4-bbb5-a357f795628d\" (UID: \"3e2b497b-98dc-4fa4-bbb5-a357f795628d\") "
Nov 04 03:43:55.677128 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:55.676952    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3e2b497b-98dc-4fa4-bbb5-a357f795628d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e2b497b-98dc-4fa4-bbb5-a357f795628d-istio-podinfo\") pod \"3e2b497b-98dc-4fa4-bbb5-a357f795628d\" (UID: \"3e2b497b-98dc-4fa4-bbb5-a357f795628d\") "
Nov 04 03:43:55.677128 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:55.676996    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3e2b497b-98dc-4fa4-bbb5-a357f795628d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e2b497b-98dc-4fa4-bbb5-a357f795628d-istio-envoy\") pod \"3e2b497b-98dc-4fa4-bbb5-a357f795628d\" (UID: \"3e2b497b-98dc-4fa4-bbb5-a357f795628d\") "
Nov 04 03:43:55.677128 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:55.677035    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3e2b497b-98dc-4fa4-bbb5-a357f795628d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e2b497b-98dc-4fa4-bbb5-a357f795628d-istiod-ca-cert\") pod \"3e2b497b-98dc-4fa4-bbb5-a357f795628d\" (UID: \"3e2b497b-98dc-4fa4-bbb5-a357f795628d\") "
Nov 04 03:43:55.677645 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:55.677614    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e2b497b-98dc-4fa4-bbb5-a357f795628d-istio-data" pod "3e2b497b-98dc-4fa4-bbb5-a357f795628d" (UID: "3e2b497b-98dc-4fa4-bbb5-a357f795628d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:43:55.677781 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:55.677660    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e2b497b-98dc-4fa4-bbb5-a357f795628d-workload-socket" pod "3e2b497b-98dc-4fa4-bbb5-a357f795628d" (UID: "3e2b497b-98dc-4fa4-bbb5-a357f795628d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:43:55.677781 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:55.677722    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e2b497b-98dc-4fa4-bbb5-a357f795628d-workload-certs" pod "3e2b497b-98dc-4fa4-bbb5-a357f795628d" (UID: "3e2b497b-98dc-4fa4-bbb5-a357f795628d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:43:55.677781 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:55.677768    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e2b497b-98dc-4fa4-bbb5-a357f795628d-istiod-ca-cert" pod "3e2b497b-98dc-4fa4-bbb5-a357f795628d" (UID: "3e2b497b-98dc-4fa4-bbb5-a357f795628d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:43:55.678068 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:55.678049    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e2b497b-98dc-4fa4-bbb5-a357f795628d-credential-socket" pod "3e2b497b-98dc-4fa4-bbb5-a357f795628d" (UID: "3e2b497b-98dc-4fa4-bbb5-a357f795628d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:43:55.679801 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:55.679776    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e2b497b-98dc-4fa4-bbb5-a357f795628d-istio-token" pod "3e2b497b-98dc-4fa4-bbb5-a357f795628d" (UID: "3e2b497b-98dc-4fa4-bbb5-a357f795628d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:43:55.679873 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:55.679796    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e2b497b-98dc-4fa4-bbb5-a357f795628d-kube-api-access-9sj7t" pod "3e2b497b-98dc-4fa4-bbb5-a357f795628d" (UID: "3e2b497b-98dc-4fa4-bbb5-a357f795628d"). InnerVolumeSpecName "kube-api-access-9sj7t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:43:55.679873 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:55.679834    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e2b497b-98dc-4fa4-bbb5-a357f795628d-istio-envoy" pod "3e2b497b-98dc-4fa4-bbb5-a357f795628d" (UID: "3e2b497b-98dc-4fa4-bbb5-a357f795628d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:43:55.679944 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:55.679875    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3e2b497b-98dc-4fa4-bbb5-a357f795628d-istio-podinfo" pod "3e2b497b-98dc-4fa4-bbb5-a357f795628d" (UID: "3e2b497b-98dc-4fa4-bbb5-a357f795628d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:43:55.836478 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:55.835917    5609 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e2b497b-98dc-4fa4-bbb5-a357f795628d-workload-certs\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:55.836478 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:55.835953    5609 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e2b497b-98dc-4fa4-bbb5-a357f795628d-istio-data\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:55.836478 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:55.835969    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9sj7t\" (UniqueName: \"kubernetes.io/projected/3e2b497b-98dc-4fa4-bbb5-a357f795628d-kube-api-access-9sj7t\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:55.836478 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:55.835984    5609 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e2b497b-98dc-4fa4-bbb5-a357f795628d-istio-token\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:55.836478 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:55.835997    5609 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e2b497b-98dc-4fa4-bbb5-a357f795628d-istio-podinfo\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:55.836478 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:55.836010    5609 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e2b497b-98dc-4fa4-bbb5-a357f795628d-istio-envoy\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:55.836478 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:55.836024    5609 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e2b497b-98dc-4fa4-bbb5-a357f795628d-istiod-ca-cert\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:55.836478 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:55.836038    5609 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e2b497b-98dc-4fa4-bbb5-a357f795628d-workload-socket\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:55.836478 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:55.836052    5609 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e2b497b-98dc-4fa4-bbb5-a357f795628d-credential-socket\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:56.490202 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:56.490167    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b"
Nov 04 03:43:56.696898 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:56.696863    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b"]
Nov 04 03:43:56.742044 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:56.741964    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-n7r6b"]
Nov 04 03:43:57.536500 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:57.536462    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3e2b497b-98dc-4fa4-bbb5-a357f795628d" path="/var/lib/kubelet/pods/3e2b497b-98dc-4fa4-bbb5-a357f795628d/volumes"
Nov 04 03:43:59.798692 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:59.798662    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-7489856dc7-h45cw"]
Nov 04 03:43:59.799157 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:59.799142    5609 cpu_manager.go:414] "containerMap: removing container" podUID="3e2b497b-98dc-4fa4-bbb5-a357f795628d" containerName="istio-proxy"
Nov 04 03:43:59.799201 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:59.799158    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3e2b497b-98dc-4fa4-bbb5-a357f795628d" containerName="istio-proxy"
Nov 04 03:43:59.799201 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:59.799182    5609 cpu_manager.go:414] "containerMap: removing container" podUID="d8178561-e4cd-488a-a88b-c62dfa4fee20" containerName="payload-pre-processing"
Nov 04 03:43:59.799201 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:59.799188    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d8178561-e4cd-488a-a88b-c62dfa4fee20" containerName="payload-pre-processing"
Nov 04 03:43:59.799201 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:59.799195    5609 cpu_manager.go:414] "containerMap: removing container" podUID="3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3" containerName="revoke-keys"
Nov 04 03:43:59.799201 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:59.799201    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3" containerName="revoke-keys"
Nov 04 03:43:59.799346 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:59.799209    5609 cpu_manager.go:414] "containerMap: removing container" podUID="72bfd449-2341-4be8-8e05-4df3fd79b8b6" containerName="istio-proxy"
Nov 04 03:43:59.799346 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:59.799214    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="72bfd449-2341-4be8-8e05-4df3fd79b8b6" containerName="istio-proxy"
Nov 04 03:43:59.799346 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:59.799297    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="3e2b497b-98dc-4fa4-bbb5-a357f795628d" containerName="istio-proxy"
Nov 04 03:43:59.799346 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:59.799308    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="d8178561-e4cd-488a-a88b-c62dfa4fee20" containerName="payload-pre-processing"
Nov 04 03:43:59.799346 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:59.799314    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3" containerName="revoke-keys"
Nov 04 03:43:59.799346 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:59.799320    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="72bfd449-2341-4be8-8e05-4df3fd79b8b6" containerName="istio-proxy"
Nov 04 03:43:59.802892 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:59.802850    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw"
Nov 04 03:43:59.902655 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:59.902611    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/16fc6217-3711-4d08-ad50-540132d280ad-provider-credentials\") pod \"payload-processing-external-model-7489856dc7-h45cw\" (UID: \"16fc6217-3711-4d08-ad50-540132d280ad\") " pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw"
Nov 04 03:43:59.902823 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:59.902685    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n78k9\" (UniqueName: \"kubernetes.io/projected/16fc6217-3711-4d08-ad50-540132d280ad-kube-api-access-n78k9\") pod \"payload-processing-external-model-7489856dc7-h45cw\" (UID: \"16fc6217-3711-4d08-ad50-540132d280ad\") " pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw"
Nov 04 03:43:59.902823 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:59.902810    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16fc6217-3711-4d08-ad50-540132d280ad-plugins-config-volume\") pod \"payload-processing-external-model-7489856dc7-h45cw\" (UID: \"16fc6217-3711-4d08-ad50-540132d280ad\") " pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw"
Nov 04 03:43:59.902899 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:59.902873    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/16fc6217-3711-4d08-ad50-540132d280ad-routing-overlay-volume\") pod \"payload-processing-external-model-7489856dc7-h45cw\" (UID: \"16fc6217-3711-4d08-ad50-540132d280ad\") " pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw"
Nov 04 03:43:59.907215 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:43:59.907184    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-7489856dc7-h45cw"]
Nov 04 03:44:00.004368 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:00.004334    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16fc6217-3711-4d08-ad50-540132d280ad-plugins-config-volume\") pod \"payload-processing-external-model-7489856dc7-h45cw\" (UID: \"16fc6217-3711-4d08-ad50-540132d280ad\") " pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw"
Nov 04 03:44:00.004535 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:00.004392    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/16fc6217-3711-4d08-ad50-540132d280ad-routing-overlay-volume\") pod \"payload-processing-external-model-7489856dc7-h45cw\" (UID: \"16fc6217-3711-4d08-ad50-540132d280ad\") " pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw"
Nov 04 03:44:00.004535 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:00.004462    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/16fc6217-3711-4d08-ad50-540132d280ad-provider-credentials\") pod \"payload-processing-external-model-7489856dc7-h45cw\" (UID: \"16fc6217-3711-4d08-ad50-540132d280ad\") " pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw"
Nov 04 03:44:00.004535 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:00.004496    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n78k9\" (UniqueName: \"kubernetes.io/projected/16fc6217-3711-4d08-ad50-540132d280ad-kube-api-access-n78k9\") pod \"payload-processing-external-model-7489856dc7-h45cw\" (UID: \"16fc6217-3711-4d08-ad50-540132d280ad\") " pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw"
Nov 04 03:44:00.005230 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:00.005199    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/16fc6217-3711-4d08-ad50-540132d280ad-routing-overlay-volume\") pod \"payload-processing-external-model-7489856dc7-h45cw\" (UID: \"16fc6217-3711-4d08-ad50-540132d280ad\") " pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw"
Nov 04 03:44:00.005348 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:00.005273    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16fc6217-3711-4d08-ad50-540132d280ad-plugins-config-volume\") pod \"payload-processing-external-model-7489856dc7-h45cw\" (UID: \"16fc6217-3711-4d08-ad50-540132d280ad\") " pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw"
Nov 04 03:44:00.006846 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:00.006820    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/16fc6217-3711-4d08-ad50-540132d280ad-provider-credentials\") pod \"payload-processing-external-model-7489856dc7-h45cw\" (UID: \"16fc6217-3711-4d08-ad50-540132d280ad\") " pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw"
Nov 04 03:44:00.071294 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:00.071229    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n78k9\" (UniqueName: \"kubernetes.io/projected/16fc6217-3711-4d08-ad50-540132d280ad-kube-api-access-n78k9\") pod \"payload-processing-external-model-7489856dc7-h45cw\" (UID: \"16fc6217-3711-4d08-ad50-540132d280ad\") " pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw"
Nov 04 03:44:00.147603 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:00.147568    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw"
Nov 04 03:44:00.526657 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:00.526619    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-7489856dc7-h45cw"]
Nov 04 03:44:00.531710 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:44:00.531675    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16fc6217_3711_4d08_ad50_540132d280ad.slice/crio-fc10dd593fd290159422914687b54c946004e2ac5ea67fa796df7818a164bb71 WatchSource:0}: Error finding container fc10dd593fd290159422914687b54c946004e2ac5ea67fa796df7818a164bb71: Status 404 returned error can't find the container with id fc10dd593fd290159422914687b54c946004e2ac5ea67fa796df7818a164bb71
Nov 04 03:44:01.535636 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:01.535596    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw" event={"ID":"16fc6217-3711-4d08-ad50-540132d280ad","Type":"ContainerStarted","Data":"34e7845f6d1bb1e0af87bb84f9f332f46cd5c86dd6b73b20b5209b16027768c6"}
Nov 04 03:44:01.536018 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:01.535650    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw"
Nov 04 03:44:01.536018 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:01.535672    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw" event={"ID":"16fc6217-3711-4d08-ad50-540132d280ad","Type":"ContainerStarted","Data":"fc10dd593fd290159422914687b54c946004e2ac5ea67fa796df7818a164bb71"}
Nov 04 03:44:01.656983 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:01.656928    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw" podStartSLOduration=2.656914112 podStartE2EDuration="2.656914112s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:43:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:44:01.655777525 +0000 UTC m=+1584.232430439" watchObservedRunningTime="2026-11-04 03:44:01.656914112 +0000 UTC m=+1584.233567026"
Nov 04 03:44:01.906487 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:01.886742    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf"]
Nov 04 03:44:01.906487 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:01.886965    5609 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf" podUID="f0e29029-1a41-41a2-bfa8-b9ecd49be65b" containerName="istio-proxy" containerID="cri-o://1c512c3da901aab3287dc3832b04f5cc472266a9e36df8c69b56b2a82e7b7e75" gracePeriod=30
Nov 04 03:44:01.920131 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:01.920098    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b62a1bcc-c6kx9"]
Nov 04 03:44:01.924747 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:01.924724    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b62a1bcc-c6kx9"
Nov 04 03:44:01.981517 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:01.981484    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x9cnx\""
Nov 04 03:44:01.981517 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:01.981481    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 03:44:02.003835 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:02.003806    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b62a1bcc-c6kx9"]
Nov 04 03:44:02.081424 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:02.076083    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/00e7f455-7909-4e74-a642-455cfd943ae1-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b62a1bcc-c6kx9\" (UID: \"00e7f455-7909-4e74-a642-455cfd943ae1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b62a1bcc-c6kx9"
Nov 04 03:44:02.081424 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:02.076162    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/00e7f455-7909-4e74-a642-455cfd943ae1-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b62a1bcc-c6kx9\" (UID: \"00e7f455-7909-4e74-a642-455cfd943ae1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b62a1bcc-c6kx9"
Nov 04 03:44:02.081424 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:02.076237    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgjbr\" (UniqueName: \"kubernetes.io/projected/00e7f455-7909-4e74-a642-455cfd943ae1-kube-api-access-kgjbr\") pod \"maas-api-revoke-sub-b62a1bcc-c6kx9\" (UID: \"00e7f455-7909-4e74-a642-455cfd943ae1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b62a1bcc-c6kx9"
Nov 04 03:44:02.176898 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:02.176868    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/00e7f455-7909-4e74-a642-455cfd943ae1-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b62a1bcc-c6kx9\" (UID: \"00e7f455-7909-4e74-a642-455cfd943ae1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b62a1bcc-c6kx9"
Nov 04 03:44:02.177056 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:02.176921    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/00e7f455-7909-4e74-a642-455cfd943ae1-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b62a1bcc-c6kx9\" (UID: \"00e7f455-7909-4e74-a642-455cfd943ae1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b62a1bcc-c6kx9"
Nov 04 03:44:02.177056 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:02.176961    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgjbr\" (UniqueName: \"kubernetes.io/projected/00e7f455-7909-4e74-a642-455cfd943ae1-kube-api-access-kgjbr\") pod \"maas-api-revoke-sub-b62a1bcc-c6kx9\" (UID: \"00e7f455-7909-4e74-a642-455cfd943ae1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b62a1bcc-c6kx9"
Nov 04 03:44:02.177548 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:02.177527    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/00e7f455-7909-4e74-a642-455cfd943ae1-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b62a1bcc-c6kx9\" (UID: \"00e7f455-7909-4e74-a642-455cfd943ae1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b62a1bcc-c6kx9"
Nov 04 03:44:02.179247 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:02.179215    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/00e7f455-7909-4e74-a642-455cfd943ae1-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b62a1bcc-c6kx9\" (UID: \"00e7f455-7909-4e74-a642-455cfd943ae1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b62a1bcc-c6kx9"
Nov 04 03:44:02.235456 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:02.235396    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgjbr\" (UniqueName: \"kubernetes.io/projected/00e7f455-7909-4e74-a642-455cfd943ae1-kube-api-access-kgjbr\") pod \"maas-api-revoke-sub-b62a1bcc-c6kx9\" (UID: \"00e7f455-7909-4e74-a642-455cfd943ae1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b62a1bcc-c6kx9"
Nov 04 03:44:02.244775 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:02.244744    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b62a1bcc-c6kx9"
Nov 04 03:44:02.637098 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:02.637012    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b62a1bcc-c6kx9"]
Nov 04 03:44:02.641265 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:44:02.641238    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00e7f455_7909_4e74_a642_455cfd943ae1.slice/crio-c6fefe38e1b92803cd6cbd1a04b4b593f3f661460b59c0acab02588f111111d3 WatchSource:0}: Error finding container c6fefe38e1b92803cd6cbd1a04b4b593f3f661460b59c0acab02588f111111d3: Status 404 returned error can't find the container with id c6fefe38e1b92803cd6cbd1a04b4b593f3f661460b59c0acab02588f111111d3
Nov 04 03:44:03.551379 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:03.551350    5609 generic.go:356] "Generic (PLEG): container finished" podID="00e7f455-7909-4e74-a642-455cfd943ae1" containerID="222acfb5920b423acbaa02b44f6734ccd0b2ee9f58de6f73bc849aa69b1c2d7f" exitCode=0
Nov 04 03:44:03.551600 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:03.551435    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b62a1bcc-c6kx9" event={"ID":"00e7f455-7909-4e74-a642-455cfd943ae1","Type":"ContainerDied","Data":"222acfb5920b423acbaa02b44f6734ccd0b2ee9f58de6f73bc849aa69b1c2d7f"}
Nov 04 03:44:03.551600 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:03.551496    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b62a1bcc-c6kx9" event={"ID":"00e7f455-7909-4e74-a642-455cfd943ae1","Type":"ContainerStarted","Data":"c6fefe38e1b92803cd6cbd1a04b4b593f3f661460b59c0acab02588f111111d3"}
Nov 04 03:44:04.738723 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:04.738694    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b62a1bcc-c6kx9"
Nov 04 03:44:04.802031 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:04.802001    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/00e7f455-7909-4e74-a642-455cfd943ae1-kube-api-access-kgjbr\" (UniqueName: \"kubernetes.io/projected/00e7f455-7909-4e74-a642-455cfd943ae1-kube-api-access-kgjbr\") pod \"00e7f455-7909-4e74-a642-455cfd943ae1\" (UID: \"00e7f455-7909-4e74-a642-455cfd943ae1\") "
Nov 04 03:44:04.802225 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:04.802053    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/00e7f455-7909-4e74-a642-455cfd943ae1-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/00e7f455-7909-4e74-a642-455cfd943ae1-maas-api-xks-ca\") pod \"00e7f455-7909-4e74-a642-455cfd943ae1\" (UID: \"00e7f455-7909-4e74-a642-455cfd943ae1\") "
Nov 04 03:44:04.802225 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:04.802134    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/00e7f455-7909-4e74-a642-455cfd943ae1-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/00e7f455-7909-4e74-a642-455cfd943ae1-maas-api-openshift-ca\") pod \"00e7f455-7909-4e74-a642-455cfd943ae1\" (UID: \"00e7f455-7909-4e74-a642-455cfd943ae1\") "
Nov 04 03:44:04.802603 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:04.802576    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00e7f455-7909-4e74-a642-455cfd943ae1-maas-api-openshift-ca" pod "00e7f455-7909-4e74-a642-455cfd943ae1" (UID: "00e7f455-7909-4e74-a642-455cfd943ae1"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:44:04.804273 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:04.804243    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00e7f455-7909-4e74-a642-455cfd943ae1-maas-api-xks-ca" pod "00e7f455-7909-4e74-a642-455cfd943ae1" (UID: "00e7f455-7909-4e74-a642-455cfd943ae1"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:44:04.804273 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:04.804261    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00e7f455-7909-4e74-a642-455cfd943ae1-kube-api-access-kgjbr" pod "00e7f455-7909-4e74-a642-455cfd943ae1" (UID: "00e7f455-7909-4e74-a642-455cfd943ae1"). InnerVolumeSpecName "kube-api-access-kgjbr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:44:04.903259 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:04.903222    5609 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/00e7f455-7909-4e74-a642-455cfd943ae1-maas-api-openshift-ca\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:04.903259 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:04.903253    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kgjbr\" (UniqueName: \"kubernetes.io/projected/00e7f455-7909-4e74-a642-455cfd943ae1-kube-api-access-kgjbr\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:04.903259 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:04.903263    5609 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/00e7f455-7909-4e74-a642-455cfd943ae1-maas-api-xks-ca\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:05.565953 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:05.565923    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b62a1bcc-c6kx9" event={"ID":"00e7f455-7909-4e74-a642-455cfd943ae1","Type":"ContainerDied","Data":"c6fefe38e1b92803cd6cbd1a04b4b593f3f661460b59c0acab02588f111111d3"}
Nov 04 03:44:05.565953 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:05.565944    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b62a1bcc-c6kx9"
Nov 04 03:44:05.566127 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:05.565968    5609 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c6fefe38e1b92803cd6cbd1a04b4b593f3f661460b59c0acab02588f111111d3"
Nov 04 03:44:07.177574 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:07.177546    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf"
Nov 04 03:44:07.230757 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:07.230676    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-kube-api-access-gg85c\" (UniqueName: \"kubernetes.io/projected/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-kube-api-access-gg85c\") pod \"f0e29029-1a41-41a2-bfa8-b9ecd49be65b\" (UID: \"f0e29029-1a41-41a2-bfa8-b9ecd49be65b\") "
Nov 04 03:44:07.230757 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:07.230721    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-istio-data\") pod \"f0e29029-1a41-41a2-bfa8-b9ecd49be65b\" (UID: \"f0e29029-1a41-41a2-bfa8-b9ecd49be65b\") "
Nov 04 03:44:07.230757 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:07.230752    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-istio-envoy\") pod \"f0e29029-1a41-41a2-bfa8-b9ecd49be65b\" (UID: \"f0e29029-1a41-41a2-bfa8-b9ecd49be65b\") "
Nov 04 03:44:07.231029 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:07.230778    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-istiod-ca-cert\") pod \"f0e29029-1a41-41a2-bfa8-b9ecd49be65b\" (UID: \"f0e29029-1a41-41a2-bfa8-b9ecd49be65b\") "
Nov 04 03:44:07.231029 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:07.230826    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-istio-token\" (UniqueName: \"kubernetes.io/projected/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-istio-token\") pod \"f0e29029-1a41-41a2-bfa8-b9ecd49be65b\" (UID: \"f0e29029-1a41-41a2-bfa8-b9ecd49be65b\") "
Nov 04 03:44:07.231029 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:07.230856    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-workload-socket\") pod \"f0e29029-1a41-41a2-bfa8-b9ecd49be65b\" (UID: \"f0e29029-1a41-41a2-bfa8-b9ecd49be65b\") "
Nov 04 03:44:07.231029 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:07.230906    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-credential-socket\") pod \"f0e29029-1a41-41a2-bfa8-b9ecd49be65b\" (UID: \"f0e29029-1a41-41a2-bfa8-b9ecd49be65b\") "
Nov 04 03:44:07.231029 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:07.230949    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-istio-podinfo\") pod \"f0e29029-1a41-41a2-bfa8-b9ecd49be65b\" (UID: \"f0e29029-1a41-41a2-bfa8-b9ecd49be65b\") "
Nov 04 03:44:07.231265 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:07.231077    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-workload-certs\") pod \"f0e29029-1a41-41a2-bfa8-b9ecd49be65b\" (UID: \"f0e29029-1a41-41a2-bfa8-b9ecd49be65b\") "
Nov 04 03:44:07.231265 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:07.231074    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-istio-data" pod "f0e29029-1a41-41a2-bfa8-b9ecd49be65b" (UID: "f0e29029-1a41-41a2-bfa8-b9ecd49be65b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:07.231365 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:07.231285    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-credential-socket" pod "f0e29029-1a41-41a2-bfa8-b9ecd49be65b" (UID: "f0e29029-1a41-41a2-bfa8-b9ecd49be65b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:07.231530 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:07.231473    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-workload-socket" pod "f0e29029-1a41-41a2-bfa8-b9ecd49be65b" (UID: "f0e29029-1a41-41a2-bfa8-b9ecd49be65b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:07.231612 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:07.231514    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-workload-certs" pod "f0e29029-1a41-41a2-bfa8-b9ecd49be65b" (UID: "f0e29029-1a41-41a2-bfa8-b9ecd49be65b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:07.231612 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:07.231539    5609 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-istio-data\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:07.231612 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:07.231558    5609 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-credential-socket\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:07.231725 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:07.231690    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-istiod-ca-cert" pod "f0e29029-1a41-41a2-bfa8-b9ecd49be65b" (UID: "f0e29029-1a41-41a2-bfa8-b9ecd49be65b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:44:07.233476 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:07.233451    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-istio-envoy" pod "f0e29029-1a41-41a2-bfa8-b9ecd49be65b" (UID: "f0e29029-1a41-41a2-bfa8-b9ecd49be65b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:07.233476 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:07.233469    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-kube-api-access-gg85c" pod "f0e29029-1a41-41a2-bfa8-b9ecd49be65b" (UID: "f0e29029-1a41-41a2-bfa8-b9ecd49be65b"). InnerVolumeSpecName "kube-api-access-gg85c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:44:07.233900 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:07.233875    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-istio-podinfo" pod "f0e29029-1a41-41a2-bfa8-b9ecd49be65b" (UID: "f0e29029-1a41-41a2-bfa8-b9ecd49be65b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:44:07.233989 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:07.233899    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-istio-token" pod "f0e29029-1a41-41a2-bfa8-b9ecd49be65b" (UID: "f0e29029-1a41-41a2-bfa8-b9ecd49be65b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:44:07.332457 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:07.332388    5609 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-workload-certs\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:07.332457 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:07.332460    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gg85c\" (UniqueName: \"kubernetes.io/projected/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-kube-api-access-gg85c\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:07.332660 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:07.332476    5609 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-istio-envoy\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:07.332660 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:07.332489    5609 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-istiod-ca-cert\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:07.332660 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:07.332501    5609 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-istio-token\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:07.332660 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:07.332514    5609 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-workload-socket\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:07.332660 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:07.332527    5609 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0e29029-1a41-41a2-bfa8-b9ecd49be65b-istio-podinfo\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:07.581492 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:07.581464    5609 generic.go:356] "Generic (PLEG): container finished" podID="f0e29029-1a41-41a2-bfa8-b9ecd49be65b" containerID="1c512c3da901aab3287dc3832b04f5cc472266a9e36df8c69b56b2a82e7b7e75" exitCode=0
Nov 04 03:44:07.581636 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:07.581508    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf"
Nov 04 03:44:07.581636 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:07.581544    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf" event={"ID":"f0e29029-1a41-41a2-bfa8-b9ecd49be65b","Type":"ContainerDied","Data":"1c512c3da901aab3287dc3832b04f5cc472266a9e36df8c69b56b2a82e7b7e75"}
Nov 04 03:44:07.581636 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:07.581604    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf" event={"ID":"f0e29029-1a41-41a2-bfa8-b9ecd49be65b","Type":"ContainerDied","Data":"187981a426d05b0b2d37529db6fbb698f5c206f811807758e26e6b57ed5f82e6"}
Nov 04 03:44:07.581636 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:07.581633    5609 scope.go:122] "RemoveContainer" containerID="1c512c3da901aab3287dc3832b04f5cc472266a9e36df8c69b56b2a82e7b7e75"
Nov 04 03:44:07.591651 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:07.591623    5609 scope.go:122] "RemoveContainer" containerID="63c87b7065c116b4a13ea75398bfbb0af83e9d050e31ef475c8507eed3dd73ec"
Nov 04 03:44:07.600602 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:07.600581    5609 scope.go:122] "RemoveContainer" containerID="1c512c3da901aab3287dc3832b04f5cc472266a9e36df8c69b56b2a82e7b7e75"
Nov 04 03:44:07.600886 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:44:07.600863    5609 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c512c3da901aab3287dc3832b04f5cc472266a9e36df8c69b56b2a82e7b7e75\": container with ID starting with 1c512c3da901aab3287dc3832b04f5cc472266a9e36df8c69b56b2a82e7b7e75 not found: ID does not exist" containerID="1c512c3da901aab3287dc3832b04f5cc472266a9e36df8c69b56b2a82e7b7e75"
Nov 04 03:44:07.600976 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:07.600900    5609 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c512c3da901aab3287dc3832b04f5cc472266a9e36df8c69b56b2a82e7b7e75"} err="failed to get container status \"1c512c3da901aab3287dc3832b04f5cc472266a9e36df8c69b56b2a82e7b7e75\": rpc error: code = NotFound desc = could not find container \"1c512c3da901aab3287dc3832b04f5cc472266a9e36df8c69b56b2a82e7b7e75\": container with ID starting with 1c512c3da901aab3287dc3832b04f5cc472266a9e36df8c69b56b2a82e7b7e75 not found: ID does not exist"
Nov 04 03:44:07.600976 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:07.600923    5609 scope.go:122] "RemoveContainer" containerID="63c87b7065c116b4a13ea75398bfbb0af83e9d050e31ef475c8507eed3dd73ec"
Nov 04 03:44:07.601144 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:44:07.601126    5609 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63c87b7065c116b4a13ea75398bfbb0af83e9d050e31ef475c8507eed3dd73ec\": container with ID starting with 63c87b7065c116b4a13ea75398bfbb0af83e9d050e31ef475c8507eed3dd73ec not found: ID does not exist" containerID="63c87b7065c116b4a13ea75398bfbb0af83e9d050e31ef475c8507eed3dd73ec"
Nov 04 03:44:07.601206 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:07.601147    5609 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63c87b7065c116b4a13ea75398bfbb0af83e9d050e31ef475c8507eed3dd73ec"} err="failed to get container status \"63c87b7065c116b4a13ea75398bfbb0af83e9d050e31ef475c8507eed3dd73ec\": rpc error: code = NotFound desc = could not find container \"63c87b7065c116b4a13ea75398bfbb0af83e9d050e31ef475c8507eed3dd73ec\": container with ID starting with 63c87b7065c116b4a13ea75398bfbb0af83e9d050e31ef475c8507eed3dd73ec not found: ID does not exist"
Nov 04 03:44:07.753783 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:07.749035    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf"]
Nov 04 03:44:07.756916 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:07.756886    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-k5vjf"]
Nov 04 03:44:09.554392 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:09.554359    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f0e29029-1a41-41a2-bfa8-b9ecd49be65b" path="/var/lib/kubelet/pods/f0e29029-1a41-41a2-bfa8-b9ecd49be65b/volumes"
Nov 04 03:44:12.545380 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:12.545337    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw" podUID="16fc6217-3711-4d08-ad50-540132d280ad" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:44:18.247374 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:18.247336    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b349fc13-fccql"]
Nov 04 03:44:18.247832 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:18.247813    5609 cpu_manager.go:414] "containerMap: removing container" podUID="00e7f455-7909-4e74-a642-455cfd943ae1" containerName="revoke-keys"
Nov 04 03:44:18.247832 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:18.247826    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="00e7f455-7909-4e74-a642-455cfd943ae1" containerName="revoke-keys"
Nov 04 03:44:18.247912 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:18.247846    5609 cpu_manager.go:414] "containerMap: removing container" podUID="f0e29029-1a41-41a2-bfa8-b9ecd49be65b" containerName="istio-proxy"
Nov 04 03:44:18.247912 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:18.247852    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f0e29029-1a41-41a2-bfa8-b9ecd49be65b" containerName="istio-proxy"
Nov 04 03:44:18.247912 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:18.247875    5609 cpu_manager.go:414] "containerMap: removing container" podUID="f0e29029-1a41-41a2-bfa8-b9ecd49be65b" containerName="istio-proxy"
Nov 04 03:44:18.247912 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:18.247880    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f0e29029-1a41-41a2-bfa8-b9ecd49be65b" containerName="istio-proxy"
Nov 04 03:44:18.248023 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:18.247949    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="00e7f455-7909-4e74-a642-455cfd943ae1" containerName="revoke-keys"
Nov 04 03:44:18.248023 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:18.247959    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="f0e29029-1a41-41a2-bfa8-b9ecd49be65b" containerName="istio-proxy"
Nov 04 03:44:18.248023 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:18.247968    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="f0e29029-1a41-41a2-bfa8-b9ecd49be65b" containerName="istio-proxy"
Nov 04 03:44:18.252666 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:18.252646    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b349fc13-fccql"
Nov 04 03:44:18.343864 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:18.343823    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kppj\" (UniqueName: \"kubernetes.io/projected/572347a8-4105-40e9-8804-e77cd582c936-kube-api-access-5kppj\") pod \"maas-api-revoke-sub-b349fc13-fccql\" (UID: \"572347a8-4105-40e9-8804-e77cd582c936\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b349fc13-fccql"
Nov 04 03:44:18.344049 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:18.343888    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/572347a8-4105-40e9-8804-e77cd582c936-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b349fc13-fccql\" (UID: \"572347a8-4105-40e9-8804-e77cd582c936\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b349fc13-fccql"
Nov 04 03:44:18.344049 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:18.344008    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/572347a8-4105-40e9-8804-e77cd582c936-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b349fc13-fccql\" (UID: \"572347a8-4105-40e9-8804-e77cd582c936\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b349fc13-fccql"
Nov 04 03:44:18.349366 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:18.349342    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 03:44:18.349512 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:18.349380    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x9cnx\""
Nov 04 03:44:18.352446 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:18.352426    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b349fc13-fccql"]
Nov 04 03:44:18.444882 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:18.444846    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/572347a8-4105-40e9-8804-e77cd582c936-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b349fc13-fccql\" (UID: \"572347a8-4105-40e9-8804-e77cd582c936\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b349fc13-fccql"
Nov 04 03:44:18.445074 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:18.444964    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/572347a8-4105-40e9-8804-e77cd582c936-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b349fc13-fccql\" (UID: \"572347a8-4105-40e9-8804-e77cd582c936\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b349fc13-fccql"
Nov 04 03:44:18.445140 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:18.445119    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kppj\" (UniqueName: \"kubernetes.io/projected/572347a8-4105-40e9-8804-e77cd582c936-kube-api-access-5kppj\") pod \"maas-api-revoke-sub-b349fc13-fccql\" (UID: \"572347a8-4105-40e9-8804-e77cd582c936\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b349fc13-fccql"
Nov 04 03:44:18.445814 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:18.445788    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/572347a8-4105-40e9-8804-e77cd582c936-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b349fc13-fccql\" (UID: \"572347a8-4105-40e9-8804-e77cd582c936\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b349fc13-fccql"
Nov 04 03:44:18.447505 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:18.447474    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/572347a8-4105-40e9-8804-e77cd582c936-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b349fc13-fccql\" (UID: \"572347a8-4105-40e9-8804-e77cd582c936\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b349fc13-fccql"
Nov 04 03:44:18.503388 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:18.503312    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kppj\" (UniqueName: \"kubernetes.io/projected/572347a8-4105-40e9-8804-e77cd582c936-kube-api-access-5kppj\") pod \"maas-api-revoke-sub-b349fc13-fccql\" (UID: \"572347a8-4105-40e9-8804-e77cd582c936\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b349fc13-fccql"
Nov 04 03:44:18.560109 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:44:18.560066    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:44:18.566376 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:18.566354    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b349fc13-fccql"
Nov 04 03:44:18.938031 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:18.937995    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b349fc13-fccql"]
Nov 04 03:44:18.942708 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:44:18.942680    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod572347a8_4105_40e9_8804_e77cd582c936.slice/crio-4d01c101a9ca8467ca0e10e3e725001bb4ec6af7e6e150e81b1c815d8ad608c4 WatchSource:0}: Error finding container 4d01c101a9ca8467ca0e10e3e725001bb4ec6af7e6e150e81b1c815d8ad608c4: Status 404 returned error can't find the container with id 4d01c101a9ca8467ca0e10e3e725001bb4ec6af7e6e150e81b1c815d8ad608c4
Nov 04 03:44:19.588774 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:19.588690    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-092f8c-7f979b7686-zq4hl"]
Nov 04 03:44:19.690903 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:19.690873    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-092f8c-679c8b7dbc-mcm7c"]
Nov 04 03:44:19.723930 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:19.723898    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b349fc13-fccql" event={"ID":"572347a8-4105-40e9-8804-e77cd582c936","Type":"ContainerStarted","Data":"e8c47d823fc13018c1f775f9c1392f80c2c23ff4610531fb65335d30c05d4790"}
Nov 04 03:44:19.724106 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:19.723946    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b349fc13-fccql" event={"ID":"572347a8-4105-40e9-8804-e77cd582c936","Type":"ContainerStarted","Data":"4d01c101a9ca8467ca0e10e3e725001bb4ec6af7e6e150e81b1c815d8ad608c4"}
Nov 04 03:44:19.920231 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:19.920103    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b349fc13-fccql" podStartSLOduration=1.9200865729999999 podStartE2EDuration="1.920086573s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:44:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:44:19.918424078 +0000 UTC m=+1602.495076988" watchObservedRunningTime="2026-11-04 03:44:19.920086573 +0000 UTC m=+1602.496739486"
Nov 04 03:44:20.735755 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:20.735726    5609 generic.go:356] "Generic (PLEG): container finished" podID="572347a8-4105-40e9-8804-e77cd582c936" containerID="e8c47d823fc13018c1f775f9c1392f80c2c23ff4610531fb65335d30c05d4790" exitCode=0
Nov 04 03:44:20.736148 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:20.735776    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b349fc13-fccql" event={"ID":"572347a8-4105-40e9-8804-e77cd582c936","Type":"ContainerDied","Data":"e8c47d823fc13018c1f775f9c1392f80c2c23ff4610531fb65335d30c05d4790"}
Nov 04 03:44:21.889201 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:21.889174    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b349fc13-fccql"
Nov 04 03:44:21.988817 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:21.988788    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/572347a8-4105-40e9-8804-e77cd582c936-kube-api-access-5kppj\" (UniqueName: \"kubernetes.io/projected/572347a8-4105-40e9-8804-e77cd582c936-kube-api-access-5kppj\") pod \"572347a8-4105-40e9-8804-e77cd582c936\" (UID: \"572347a8-4105-40e9-8804-e77cd582c936\") "
Nov 04 03:44:21.988997 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:21.988973    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/572347a8-4105-40e9-8804-e77cd582c936-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/572347a8-4105-40e9-8804-e77cd582c936-maas-api-openshift-ca\") pod \"572347a8-4105-40e9-8804-e77cd582c936\" (UID: \"572347a8-4105-40e9-8804-e77cd582c936\") "
Nov 04 03:44:21.989069 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:21.989025    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/572347a8-4105-40e9-8804-e77cd582c936-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/572347a8-4105-40e9-8804-e77cd582c936-maas-api-xks-ca\") pod \"572347a8-4105-40e9-8804-e77cd582c936\" (UID: \"572347a8-4105-40e9-8804-e77cd582c936\") "
Nov 04 03:44:21.989380 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:21.989352    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/572347a8-4105-40e9-8804-e77cd582c936-maas-api-openshift-ca" pod "572347a8-4105-40e9-8804-e77cd582c936" (UID: "572347a8-4105-40e9-8804-e77cd582c936"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:44:21.990991 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:21.990967    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/572347a8-4105-40e9-8804-e77cd582c936-kube-api-access-5kppj" pod "572347a8-4105-40e9-8804-e77cd582c936" (UID: "572347a8-4105-40e9-8804-e77cd582c936"). InnerVolumeSpecName "kube-api-access-5kppj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:44:21.991378 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:21.991360    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/572347a8-4105-40e9-8804-e77cd582c936-maas-api-xks-ca" pod "572347a8-4105-40e9-8804-e77cd582c936" (UID: "572347a8-4105-40e9-8804-e77cd582c936"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:44:22.090897 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:22.090864    5609 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/572347a8-4105-40e9-8804-e77cd582c936-maas-api-openshift-ca\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:22.090897 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:22.090895    5609 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/572347a8-4105-40e9-8804-e77cd582c936-maas-api-xks-ca\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:22.091113 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:22.090905    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5kppj\" (UniqueName: \"kubernetes.io/projected/572347a8-4105-40e9-8804-e77cd582c936-kube-api-access-5kppj\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:22.544033 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:22.543995    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw" podUID="16fc6217-3711-4d08-ad50-540132d280ad" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:44:22.752485 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:22.752451    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b349fc13-fccql" event={"ID":"572347a8-4105-40e9-8804-e77cd582c936","Type":"ContainerDied","Data":"4d01c101a9ca8467ca0e10e3e725001bb4ec6af7e6e150e81b1c815d8ad608c4"}
Nov 04 03:44:22.752485 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:22.752478    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b349fc13-fccql"
Nov 04 03:44:22.752678 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:22.752497    5609 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4d01c101a9ca8467ca0e10e3e725001bb4ec6af7e6e150e81b1c815d8ad608c4"
Nov 04 03:44:24.589657 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:24.589578    5609 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-092f8c-7f979b7686-zq4hl" podUID="cf09732a-b411-43d6-81dc-0240f879a783" containerName="payload-processing" containerID="cri-o://5c77b6d26398d0c21b00cdbe7bed4203097ca2ebde46ecc7a4ea6d8d1abd6630" gracePeriod=25
Nov 04 03:44:24.692011 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:24.691953    5609 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-092f8c-679c8b7dbc-mcm7c" podUID="2e3ba585-0eb0-44bb-955c-3b07f75123f9" containerName="payload-pre-processing" containerID="cri-o://e3dbebc453dee4a2c586f564b27f2ce58892cafd8a1a8d83c83d63542d940ec5" gracePeriod=25
Nov 04 03:44:28.724773 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:28.724732    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-092f8c-679c8b7dbc-mcm7c" podUID="2e3ba585-0eb0-44bb-955c-3b07f75123f9" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:44:29.731931 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:29.731895    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w0-092f8c-7f979b7686-zq4hl" podUID="cf09732a-b411-43d6-81dc-0240f879a783" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:44:32.543283 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:32.543247    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw" podUID="16fc6217-3711-4d08-ad50-540132d280ad" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:44:35.250180 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:35.250144    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng"]
Nov 04 03:44:35.250733 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:35.250717    5609 cpu_manager.go:414] "containerMap: removing container" podUID="572347a8-4105-40e9-8804-e77cd582c936" containerName="revoke-keys"
Nov 04 03:44:35.250840 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:35.250734    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="572347a8-4105-40e9-8804-e77cd582c936" containerName="revoke-keys"
Nov 04 03:44:35.250892 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:35.250850    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="572347a8-4105-40e9-8804-e77cd582c936" containerName="revoke-keys"
Nov 04 03:44:35.254167 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:35.254148    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng"
Nov 04 03:44:35.329698 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:35.329661    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf447f1-dd36-4897-9a3d-9fdcd71cf288-workload-socket\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng\" (UID: \"acf447f1-dd36-4897-9a3d-9fdcd71cf288\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng"
Nov 04 03:44:35.329857 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:35.329710    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acf447f1-dd36-4897-9a3d-9fdcd71cf288-istiod-ca-cert\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng\" (UID: \"acf447f1-dd36-4897-9a3d-9fdcd71cf288\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng"
Nov 04 03:44:35.329857 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:35.329810    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acf447f1-dd36-4897-9a3d-9fdcd71cf288-istio-data\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng\" (UID: \"acf447f1-dd36-4897-9a3d-9fdcd71cf288\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng"
Nov 04 03:44:35.329938 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:35.329874    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acf447f1-dd36-4897-9a3d-9fdcd71cf288-istio-podinfo\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng\" (UID: \"acf447f1-dd36-4897-9a3d-9fdcd71cf288\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng"
Nov 04 03:44:35.329938 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:35.329894    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf447f1-dd36-4897-9a3d-9fdcd71cf288-credential-socket\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng\" (UID: \"acf447f1-dd36-4897-9a3d-9fdcd71cf288\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng"
Nov 04 03:44:35.330000 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:35.329947    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acf447f1-dd36-4897-9a3d-9fdcd71cf288-workload-certs\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng\" (UID: \"acf447f1-dd36-4897-9a3d-9fdcd71cf288\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng"
Nov 04 03:44:35.330000 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:35.329990    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acf447f1-dd36-4897-9a3d-9fdcd71cf288-istio-envoy\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng\" (UID: \"acf447f1-dd36-4897-9a3d-9fdcd71cf288\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng"
Nov 04 03:44:35.330073 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:35.330018    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxwfk\" (UniqueName: \"kubernetes.io/projected/acf447f1-dd36-4897-9a3d-9fdcd71cf288-kube-api-access-wxwfk\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng\" (UID: \"acf447f1-dd36-4897-9a3d-9fdcd71cf288\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng"
Nov 04 03:44:35.330107 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:35.330072    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acf447f1-dd36-4897-9a3d-9fdcd71cf288-istio-token\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng\" (UID: \"acf447f1-dd36-4897-9a3d-9fdcd71cf288\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng"
Nov 04 03:44:35.431001 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:35.430961    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acf447f1-dd36-4897-9a3d-9fdcd71cf288-istio-podinfo\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng\" (UID: \"acf447f1-dd36-4897-9a3d-9fdcd71cf288\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng"
Nov 04 03:44:35.431001 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:35.431005    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf447f1-dd36-4897-9a3d-9fdcd71cf288-credential-socket\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng\" (UID: \"acf447f1-dd36-4897-9a3d-9fdcd71cf288\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng"
Nov 04 03:44:35.431275 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:35.431027    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acf447f1-dd36-4897-9a3d-9fdcd71cf288-workload-certs\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng\" (UID: \"acf447f1-dd36-4897-9a3d-9fdcd71cf288\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng"
Nov 04 03:44:35.431275 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:35.431075    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acf447f1-dd36-4897-9a3d-9fdcd71cf288-istio-envoy\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng\" (UID: \"acf447f1-dd36-4897-9a3d-9fdcd71cf288\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng"
Nov 04 03:44:35.431275 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:35.431109    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxwfk\" (UniqueName: \"kubernetes.io/projected/acf447f1-dd36-4897-9a3d-9fdcd71cf288-kube-api-access-wxwfk\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng\" (UID: \"acf447f1-dd36-4897-9a3d-9fdcd71cf288\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng"
Nov 04 03:44:35.431275 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:35.431136    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acf447f1-dd36-4897-9a3d-9fdcd71cf288-istio-token\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng\" (UID: \"acf447f1-dd36-4897-9a3d-9fdcd71cf288\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng"
Nov 04 03:44:35.431275 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:35.431218    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf447f1-dd36-4897-9a3d-9fdcd71cf288-workload-socket\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng\" (UID: \"acf447f1-dd36-4897-9a3d-9fdcd71cf288\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng"
Nov 04 03:44:35.431275 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:35.431260    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acf447f1-dd36-4897-9a3d-9fdcd71cf288-istiod-ca-cert\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng\" (UID: \"acf447f1-dd36-4897-9a3d-9fdcd71cf288\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng"
Nov 04 03:44:35.431599 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:35.431306    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acf447f1-dd36-4897-9a3d-9fdcd71cf288-istio-data\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng\" (UID: \"acf447f1-dd36-4897-9a3d-9fdcd71cf288\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng"
Nov 04 03:44:35.431599 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:35.431472    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf447f1-dd36-4897-9a3d-9fdcd71cf288-credential-socket\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng\" (UID: \"acf447f1-dd36-4897-9a3d-9fdcd71cf288\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng"
Nov 04 03:44:35.431599 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:35.431540    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acf447f1-dd36-4897-9a3d-9fdcd71cf288-workload-certs\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng\" (UID: \"acf447f1-dd36-4897-9a3d-9fdcd71cf288\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng"
Nov 04 03:44:35.431749 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:35.431623    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acf447f1-dd36-4897-9a3d-9fdcd71cf288-istio-data\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng\" (UID: \"acf447f1-dd36-4897-9a3d-9fdcd71cf288\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng"
Nov 04 03:44:35.431749 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:35.431716    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf447f1-dd36-4897-9a3d-9fdcd71cf288-workload-socket\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng\" (UID: \"acf447f1-dd36-4897-9a3d-9fdcd71cf288\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng"
Nov 04 03:44:35.432357 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:35.432332    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acf447f1-dd36-4897-9a3d-9fdcd71cf288-istiod-ca-cert\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng\" (UID: \"acf447f1-dd36-4897-9a3d-9fdcd71cf288\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng"
Nov 04 03:44:35.433655 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:35.433632    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acf447f1-dd36-4897-9a3d-9fdcd71cf288-istio-envoy\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng\" (UID: \"acf447f1-dd36-4897-9a3d-9fdcd71cf288\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng"
Nov 04 03:44:35.434024 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:35.434003    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acf447f1-dd36-4897-9a3d-9fdcd71cf288-istio-podinfo\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng\" (UID: \"acf447f1-dd36-4897-9a3d-9fdcd71cf288\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng"
Nov 04 03:44:35.461205 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:35.461172    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng"]
Nov 04 03:44:35.526188 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:35.526050    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxwfk\" (UniqueName: \"kubernetes.io/projected/acf447f1-dd36-4897-9a3d-9fdcd71cf288-kube-api-access-wxwfk\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng\" (UID: \"acf447f1-dd36-4897-9a3d-9fdcd71cf288\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng"
Nov 04 03:44:35.568916 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:35.567480    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acf447f1-dd36-4897-9a3d-9fdcd71cf288-istio-token\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng\" (UID: \"acf447f1-dd36-4897-9a3d-9fdcd71cf288\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng"
Nov 04 03:44:35.568916 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:35.567691    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng"
Nov 04 03:44:36.014013 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:36.013985    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng"]
Nov 04 03:44:36.017644 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:44:36.017616    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacf447f1_dd36_4897_9a3d_9fdcd71cf288.slice/crio-41f773ded09a869acecc039e92d92d44d89da2ca340aec573af58e5e13a7585d WatchSource:0}: Error finding container 41f773ded09a869acecc039e92d92d44d89da2ca340aec573af58e5e13a7585d: Status 404 returned error can't find the container with id 41f773ded09a869acecc039e92d92d44d89da2ca340aec573af58e5e13a7585d
Nov 04 03:44:36.020458 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:36.020416    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:44:36.020543 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:36.020502    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:44:36.020582 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:36.020540    5609 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 03:44:36.845032 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:36.844989    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng" event={"ID":"acf447f1-dd36-4897-9a3d-9fdcd71cf288","Type":"ContainerStarted","Data":"7434e10cb1473596a213e602e936076b778c999202b4f560b994dc26082981b4"}
Nov 04 03:44:36.845539 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:36.845049    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng" event={"ID":"acf447f1-dd36-4897-9a3d-9fdcd71cf288","Type":"ContainerStarted","Data":"41f773ded09a869acecc039e92d92d44d89da2ca340aec573af58e5e13a7585d"}
Nov 04 03:44:37.072752 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:37.072705    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng" podStartSLOduration=2.072692221 podStartE2EDuration="2.072692221s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:44:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:44:37.023226264 +0000 UTC m=+1619.599879181" watchObservedRunningTime="2026-11-04 03:44:37.072692221 +0000 UTC m=+1619.649345134"
Nov 04 03:44:37.568678 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:37.568648    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng"
Nov 04 03:44:37.570437 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:37.570392    5609 patch_prober.go:29] interesting pod/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Nov 04 03:44:37.570559 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:37.570461    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng" podUID="acf447f1-dd36-4897-9a3d-9fdcd71cf288" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Nov 04 03:44:38.035807 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:38.035768    5609 scope.go:122] "RemoveContainer" containerID="52113e6d5858fda7d97ed01e115b705fc8dcad081c482d8bfdcb0129e97fd285"
Nov 04 03:44:38.568767 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:38.568734    5609 patch_prober.go:29] interesting pod/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Nov 04 03:44:38.568949 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:38.568799    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng" podUID="acf447f1-dd36-4897-9a3d-9fdcd71cf288" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Nov 04 03:44:38.724704 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:38.724661    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-092f8c-679c8b7dbc-mcm7c" podUID="2e3ba585-0eb0-44bb-955c-3b07f75123f9" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:44:39.569053 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:39.569018    5609 patch_prober.go:29] interesting pod/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Nov 04 03:44:39.569478 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:39.569071    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng" podUID="acf447f1-dd36-4897-9a3d-9fdcd71cf288" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Nov 04 03:44:39.731079 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:39.731039    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w0-092f8c-7f979b7686-zq4hl" podUID="cf09732a-b411-43d6-81dc-0240f879a783" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:44:40.569000 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:40.568962    5609 patch_prober.go:29] interesting pod/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Nov 04 03:44:40.569187 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:40.569016    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng" podUID="acf447f1-dd36-4897-9a3d-9fdcd71cf288" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Nov 04 03:44:41.574430 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:41.571812    5609 patch_prober.go:29] interesting pod/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Nov 04 03:44:41.574430 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:41.571874    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng" podUID="acf447f1-dd36-4897-9a3d-9fdcd71cf288" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Nov 04 03:44:41.654585 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:44:41.654549    5609 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-8b97550813bf: secret "agc-cred-8b97550813bf" not found
Nov 04 03:44:41.654585 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:44:41.654581    5609 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-df4cf9975-hk75w: secret "agc-cred-8b97550813bf" not found
Nov 04 03:44:41.654845 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:44:41.654657    5609 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d-provider-credentials podName:0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d nodeName:}" failed. No retries permitted until 2026-11-04 03:44:42.154634245 +0000 UTC m=+1624.731287157 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d-provider-credentials") pod "payload-processing-external-model-df4cf9975-hk75w" (UID: "0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d") : secret "agc-cred-8b97550813bf" not found
Nov 04 03:44:42.158985 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:44:42.158948    5609 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-8b97550813bf: secret "agc-cred-8b97550813bf" not found
Nov 04 03:44:42.158985 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:44:42.158975    5609 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-df4cf9975-hk75w: secret "agc-cred-8b97550813bf" not found
Nov 04 03:44:42.159218 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:44:42.159045    5609 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d-provider-credentials podName:0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d nodeName:}" failed. No retries permitted until 2026-11-04 03:44:43.159026447 +0000 UTC m=+1625.735679352 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d-provider-credentials") pod "payload-processing-external-model-df4cf9975-hk75w" (UID: "0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d") : secret "agc-cred-8b97550813bf" not found
Nov 04 03:44:42.543356 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:42.543322    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw" podUID="16fc6217-3711-4d08-ad50-540132d280ad" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:44:42.568718 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:42.568683    5609 patch_prober.go:29] interesting pod/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Nov 04 03:44:42.568868 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:42.568737    5609 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng" podUID="acf447f1-dd36-4897-9a3d-9fdcd71cf288" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Nov 04 03:44:43.172969 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:44:43.172911    5609 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-8b97550813bf: secret "agc-cred-8b97550813bf" not found
Nov 04 03:44:43.172969 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:44:43.172964    5609 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-df4cf9975-hk75w: secret "agc-cred-8b97550813bf" not found
Nov 04 03:44:43.173528 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:44:43.173046    5609 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d-provider-credentials podName:0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d nodeName:}" failed. No retries permitted until 2026-11-04 03:44:45.173021539 +0000 UTC m=+1627.749674452 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d-provider-credentials") pod "payload-processing-external-model-df4cf9975-hk75w" (UID: "0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d") : secret "agc-cred-8b97550813bf" not found
Nov 04 03:44:43.834117 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:43.834084    5609 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng"
Nov 04 03:44:43.834674 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:43.834635    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng"
Nov 04 03:44:43.835693 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:43.835665    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng"
Nov 04 03:44:44.862051 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:44.861978    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-092f8c-7f979b7686-zq4hl"
Nov 04 03:44:44.919041 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:44.919010    5609 generic.go:356] "Generic (PLEG): container finished" podID="2e3ba585-0eb0-44bb-955c-3b07f75123f9" containerID="e3dbebc453dee4a2c586f564b27f2ce58892cafd8a1a8d83c83d63542d940ec5" exitCode=0
Nov 04 03:44:44.919185 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:44.919075    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-092f8c-679c8b7dbc-mcm7c" event={"ID":"2e3ba585-0eb0-44bb-955c-3b07f75123f9","Type":"ContainerDied","Data":"e3dbebc453dee4a2c586f564b27f2ce58892cafd8a1a8d83c83d63542d940ec5"}
Nov 04 03:44:44.920759 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:44.920734    5609 generic.go:356] "Generic (PLEG): container finished" podID="cf09732a-b411-43d6-81dc-0240f879a783" containerID="5c77b6d26398d0c21b00cdbe7bed4203097ca2ebde46ecc7a4ea6d8d1abd6630" exitCode=0
Nov 04 03:44:44.920904 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:44.920767    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-092f8c-7f979b7686-zq4hl" event={"ID":"cf09732a-b411-43d6-81dc-0240f879a783","Type":"ContainerDied","Data":"5c77b6d26398d0c21b00cdbe7bed4203097ca2ebde46ecc7a4ea6d8d1abd6630"}
Nov 04 03:44:44.920904 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:44.920787    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-092f8c-7f979b7686-zq4hl"
Nov 04 03:44:44.920904 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:44.920826    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-092f8c-7f979b7686-zq4hl" event={"ID":"cf09732a-b411-43d6-81dc-0240f879a783","Type":"ContainerDied","Data":"1d01b311b87b9b7c035d9df76c899c81d7e3d150f1b0aa5575979b617df65831"}
Nov 04 03:44:44.920904 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:44.920858    5609 scope.go:122] "RemoveContainer" containerID="5c77b6d26398d0c21b00cdbe7bed4203097ca2ebde46ecc7a4ea6d8d1abd6630"
Nov 04 03:44:44.930920 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:44.930896    5609 scope.go:122] "RemoveContainer" containerID="5c77b6d26398d0c21b00cdbe7bed4203097ca2ebde46ecc7a4ea6d8d1abd6630"
Nov 04 03:44:44.931191 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:44:44.931164    5609 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c77b6d26398d0c21b00cdbe7bed4203097ca2ebde46ecc7a4ea6d8d1abd6630\": container with ID starting with 5c77b6d26398d0c21b00cdbe7bed4203097ca2ebde46ecc7a4ea6d8d1abd6630 not found: ID does not exist" containerID="5c77b6d26398d0c21b00cdbe7bed4203097ca2ebde46ecc7a4ea6d8d1abd6630"
Nov 04 03:44:44.931270 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:44.931195    5609 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c77b6d26398d0c21b00cdbe7bed4203097ca2ebde46ecc7a4ea6d8d1abd6630"} err="failed to get container status \"5c77b6d26398d0c21b00cdbe7bed4203097ca2ebde46ecc7a4ea6d8d1abd6630\": rpc error: code = NotFound desc = could not find container \"5c77b6d26398d0c21b00cdbe7bed4203097ca2ebde46ecc7a4ea6d8d1abd6630\": container with ID starting with 5c77b6d26398d0c21b00cdbe7bed4203097ca2ebde46ecc7a4ea6d8d1abd6630 not found: ID does not exist"
Nov 04 03:44:44.953318 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:44.953300    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-092f8c-679c8b7dbc-mcm7c"
Nov 04 03:44:44.993803 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:44.993773    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf09732a-b411-43d6-81dc-0240f879a783-kube-api-access-7lg8j\" (UniqueName: \"kubernetes.io/projected/cf09732a-b411-43d6-81dc-0240f879a783-kube-api-access-7lg8j\") pod \"cf09732a-b411-43d6-81dc-0240f879a783\" (UID: \"cf09732a-b411-43d6-81dc-0240f879a783\") "
Nov 04 03:44:44.993985 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:44.993908    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cf09732a-b411-43d6-81dc-0240f879a783-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf09732a-b411-43d6-81dc-0240f879a783-plugins-config-volume\") pod \"cf09732a-b411-43d6-81dc-0240f879a783\" (UID: \"cf09732a-b411-43d6-81dc-0240f879a783\") "
Nov 04 03:44:44.994311 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:44.994284    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf09732a-b411-43d6-81dc-0240f879a783-plugins-config-volume" pod "cf09732a-b411-43d6-81dc-0240f879a783" (UID: "cf09732a-b411-43d6-81dc-0240f879a783"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:44:44.994801 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:44.994776    5609 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf09732a-b411-43d6-81dc-0240f879a783-plugins-config-volume\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:44.996070 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:44.996043    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf09732a-b411-43d6-81dc-0240f879a783-kube-api-access-7lg8j" pod "cf09732a-b411-43d6-81dc-0240f879a783" (UID: "cf09732a-b411-43d6-81dc-0240f879a783"). InnerVolumeSpecName "kube-api-access-7lg8j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:44:45.095651 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:45.095615    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2e3ba585-0eb0-44bb-955c-3b07f75123f9-kube-api-access-krx9r\" (UniqueName: \"kubernetes.io/projected/2e3ba585-0eb0-44bb-955c-3b07f75123f9-kube-api-access-krx9r\") pod \"2e3ba585-0eb0-44bb-955c-3b07f75123f9\" (UID: \"2e3ba585-0eb0-44bb-955c-3b07f75123f9\") "
Nov 04 03:44:45.095873 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:45.095715    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2e3ba585-0eb0-44bb-955c-3b07f75123f9-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e3ba585-0eb0-44bb-955c-3b07f75123f9-plugins-config-volume\") pod \"2e3ba585-0eb0-44bb-955c-3b07f75123f9\" (UID: \"2e3ba585-0eb0-44bb-955c-3b07f75123f9\") "
Nov 04 03:44:45.095975 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:45.095961    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7lg8j\" (UniqueName: \"kubernetes.io/projected/cf09732a-b411-43d6-81dc-0240f879a783-kube-api-access-7lg8j\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:45.096125 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:45.096102    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e3ba585-0eb0-44bb-955c-3b07f75123f9-plugins-config-volume" pod "2e3ba585-0eb0-44bb-955c-3b07f75123f9" (UID: "2e3ba585-0eb0-44bb-955c-3b07f75123f9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:44:45.097713 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:45.097692    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e3ba585-0eb0-44bb-955c-3b07f75123f9-kube-api-access-krx9r" pod "2e3ba585-0eb0-44bb-955c-3b07f75123f9" (UID: "2e3ba585-0eb0-44bb-955c-3b07f75123f9"). InnerVolumeSpecName "kube-api-access-krx9r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:44:45.196977 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:45.196946    5609 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e3ba585-0eb0-44bb-955c-3b07f75123f9-plugins-config-volume\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:45.196977 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:45.196971    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-krx9r\" (UniqueName: \"kubernetes.io/projected/2e3ba585-0eb0-44bb-955c-3b07f75123f9-kube-api-access-krx9r\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:45.197179 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:44:45.197016    5609 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-8b97550813bf: secret "agc-cred-8b97550813bf" not found
Nov 04 03:44:45.197179 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:44:45.197040    5609 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-df4cf9975-hk75w: secret "agc-cred-8b97550813bf" not found
Nov 04 03:44:45.197179 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:44:45.197119    5609 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d-provider-credentials podName:0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d nodeName:}" failed. No retries permitted until 2026-11-04 03:44:49.197099257 +0000 UTC m=+1631.773752155 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d-provider-credentials") pod "payload-processing-external-model-df4cf9975-hk75w" (UID: "0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d") : secret "agc-cred-8b97550813bf" not found
Nov 04 03:44:45.337904 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:45.337860    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-092f8c-7f979b7686-zq4hl"]
Nov 04 03:44:45.341757 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:45.341730    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-092f8c-7f979b7686-zq4hl"]
Nov 04 03:44:45.535974 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:45.535888    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cf09732a-b411-43d6-81dc-0240f879a783" path="/var/lib/kubelet/pods/cf09732a-b411-43d6-81dc-0240f879a783/volumes"
Nov 04 03:44:45.799858 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:45.799765    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng"]
Nov 04 03:44:45.821252 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:45.821215    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x"]
Nov 04 03:44:45.821517 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:45.821492    5609 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x" podUID="66120169-8cf1-4166-93b9-2ae671872165" containerName="istio-proxy" containerID="cri-o://2cad8d652e639b0c844641145cb8345e20739565d7838e050b2a11e4584e2313" gracePeriod=30
Nov 04 03:44:45.954750 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:45.954709    5609 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng" podUID="acf447f1-dd36-4897-9a3d-9fdcd71cf288" containerName="istio-proxy" containerID="cri-o://7434e10cb1473596a213e602e936076b778c999202b4f560b994dc26082981b4" gracePeriod=30
Nov 04 03:44:45.955941 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:45.955379    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-092f8c-679c8b7dbc-mcm7c" event={"ID":"2e3ba585-0eb0-44bb-955c-3b07f75123f9","Type":"ContainerDied","Data":"8696c14f1cf67a31b83cb5b19731c6dc7a2090dbd76282b2abed3f4abb22670f"}
Nov 04 03:44:45.956100 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:45.955976    5609 scope.go:122] "RemoveContainer" containerID="e3dbebc453dee4a2c586f564b27f2ce58892cafd8a1a8d83c83d63542d940ec5"
Nov 04 03:44:45.956100 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:45.955455    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-092f8c-679c8b7dbc-mcm7c"
Nov 04 03:44:46.180123 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:46.180099    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-092f8c-679c8b7dbc-mcm7c"]
Nov 04 03:44:46.186526 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:46.186500    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-092f8c-679c8b7dbc-mcm7c"]
Nov 04 03:44:47.534208 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:47.534165    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2e3ba585-0eb0-44bb-955c-3b07f75123f9" path="/var/lib/kubelet/pods/2e3ba585-0eb0-44bb-955c-3b07f75123f9/volumes"
Nov 04 03:44:48.559143 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:44:48.559110    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:44:49.258671 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:44:49.258629    5609 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-8b97550813bf: secret "agc-cred-8b97550813bf" not found
Nov 04 03:44:49.258671 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:44:49.258664    5609 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-df4cf9975-hk75w: secret "agc-cred-8b97550813bf" not found
Nov 04 03:44:49.258888 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:44:49.258748    5609 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d-provider-credentials podName:0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d nodeName:}" failed. No retries permitted until 2026-11-04 03:44:57.258725282 +0000 UTC m=+1639.835378178 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d-provider-credentials") pod "payload-processing-external-model-df4cf9975-hk75w" (UID: "0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d") : secret "agc-cred-8b97550813bf" not found
Nov 04 03:44:50.300178 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:50.300142    5609 patch_prober.go:29] interesting pod/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 03:44:50.300575 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:50.300211    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x" podUID="66120169-8cf1-4166-93b9-2ae671872165" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 03:44:51.001715 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:50.990469    5609 generic.go:356] "Generic (PLEG): container finished" podID="66120169-8cf1-4166-93b9-2ae671872165" containerID="2cad8d652e639b0c844641145cb8345e20739565d7838e050b2a11e4584e2313" exitCode=0
Nov 04 03:44:51.001715 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:50.990539    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x" event={"ID":"66120169-8cf1-4166-93b9-2ae671872165","Type":"ContainerDied","Data":"2cad8d652e639b0c844641145cb8345e20739565d7838e050b2a11e4584e2313"}
Nov 04 03:44:51.001715 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:50.990601    5609 scope.go:122] "RemoveContainer" containerID="cb7dfa2d744d7fc29afec6722ed2b80825b2dcb49cfec75ad23d31ada434df12"
Nov 04 03:44:51.091130 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.091105    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x"
Nov 04 03:44:51.216582 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.216554    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng"
Nov 04 03:44:51.281391 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.281363    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/66120169-8cf1-4166-93b9-2ae671872165-kube-api-access-jphnr\" (UniqueName: \"kubernetes.io/projected/66120169-8cf1-4166-93b9-2ae671872165-kube-api-access-jphnr\") pod \"66120169-8cf1-4166-93b9-2ae671872165\" (UID: \"66120169-8cf1-4166-93b9-2ae671872165\") "
Nov 04 03:44:51.281591 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.281428    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/66120169-8cf1-4166-93b9-2ae671872165-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/66120169-8cf1-4166-93b9-2ae671872165-workload-certs\") pod \"66120169-8cf1-4166-93b9-2ae671872165\" (UID: \"66120169-8cf1-4166-93b9-2ae671872165\") "
Nov 04 03:44:51.281591 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.281479    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/66120169-8cf1-4166-93b9-2ae671872165-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/66120169-8cf1-4166-93b9-2ae671872165-istio-envoy\") pod \"66120169-8cf1-4166-93b9-2ae671872165\" (UID: \"66120169-8cf1-4166-93b9-2ae671872165\") "
Nov 04 03:44:51.281591 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.281523    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/66120169-8cf1-4166-93b9-2ae671872165-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/66120169-8cf1-4166-93b9-2ae671872165-credential-socket\") pod \"66120169-8cf1-4166-93b9-2ae671872165\" (UID: \"66120169-8cf1-4166-93b9-2ae671872165\") "
Nov 04 03:44:51.281787 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.281596    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/66120169-8cf1-4166-93b9-2ae671872165-istio-token\" (UniqueName: \"kubernetes.io/projected/66120169-8cf1-4166-93b9-2ae671872165-istio-token\") pod \"66120169-8cf1-4166-93b9-2ae671872165\" (UID: \"66120169-8cf1-4166-93b9-2ae671872165\") "
Nov 04 03:44:51.281787 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.281637    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/66120169-8cf1-4166-93b9-2ae671872165-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/66120169-8cf1-4166-93b9-2ae671872165-istio-podinfo\") pod \"66120169-8cf1-4166-93b9-2ae671872165\" (UID: \"66120169-8cf1-4166-93b9-2ae671872165\") "
Nov 04 03:44:51.281787 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.281694    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/66120169-8cf1-4166-93b9-2ae671872165-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/66120169-8cf1-4166-93b9-2ae671872165-istiod-ca-cert\") pod \"66120169-8cf1-4166-93b9-2ae671872165\" (UID: \"66120169-8cf1-4166-93b9-2ae671872165\") "
Nov 04 03:44:51.281787 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.281727    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/66120169-8cf1-4166-93b9-2ae671872165-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/66120169-8cf1-4166-93b9-2ae671872165-workload-socket\") pod \"66120169-8cf1-4166-93b9-2ae671872165\" (UID: \"66120169-8cf1-4166-93b9-2ae671872165\") "
Nov 04 03:44:51.281787 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.281759    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/66120169-8cf1-4166-93b9-2ae671872165-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/66120169-8cf1-4166-93b9-2ae671872165-istio-data\") pod \"66120169-8cf1-4166-93b9-2ae671872165\" (UID: \"66120169-8cf1-4166-93b9-2ae671872165\") "
Nov 04 03:44:51.281787 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.281771    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66120169-8cf1-4166-93b9-2ae671872165-workload-certs" pod "66120169-8cf1-4166-93b9-2ae671872165" (UID: "66120169-8cf1-4166-93b9-2ae671872165"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:51.282622 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.282090    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66120169-8cf1-4166-93b9-2ae671872165-istiod-ca-cert" pod "66120169-8cf1-4166-93b9-2ae671872165" (UID: "66120169-8cf1-4166-93b9-2ae671872165"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:44:51.282622 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.282301    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66120169-8cf1-4166-93b9-2ae671872165-istio-data" pod "66120169-8cf1-4166-93b9-2ae671872165" (UID: "66120169-8cf1-4166-93b9-2ae671872165"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:51.282622 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.282376    5609 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/66120169-8cf1-4166-93b9-2ae671872165-istiod-ca-cert\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:51.282622 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.282415    5609 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/66120169-8cf1-4166-93b9-2ae671872165-workload-certs\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:51.282622 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.282453    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66120169-8cf1-4166-93b9-2ae671872165-credential-socket" pod "66120169-8cf1-4166-93b9-2ae671872165" (UID: "66120169-8cf1-4166-93b9-2ae671872165"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:51.282622 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.282582    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66120169-8cf1-4166-93b9-2ae671872165-workload-socket" pod "66120169-8cf1-4166-93b9-2ae671872165" (UID: "66120169-8cf1-4166-93b9-2ae671872165"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:51.284137 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.284111    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66120169-8cf1-4166-93b9-2ae671872165-kube-api-access-jphnr" pod "66120169-8cf1-4166-93b9-2ae671872165" (UID: "66120169-8cf1-4166-93b9-2ae671872165"). InnerVolumeSpecName "kube-api-access-jphnr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:44:51.284628 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.284601    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/66120169-8cf1-4166-93b9-2ae671872165-istio-podinfo" pod "66120169-8cf1-4166-93b9-2ae671872165" (UID: "66120169-8cf1-4166-93b9-2ae671872165"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:44:51.284727 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.284673    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66120169-8cf1-4166-93b9-2ae671872165-istio-envoy" pod "66120169-8cf1-4166-93b9-2ae671872165" (UID: "66120169-8cf1-4166-93b9-2ae671872165"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:51.284885 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.284864    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66120169-8cf1-4166-93b9-2ae671872165-istio-token" pod "66120169-8cf1-4166-93b9-2ae671872165" (UID: "66120169-8cf1-4166-93b9-2ae671872165"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:44:51.383700 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.383622    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/acf447f1-dd36-4897-9a3d-9fdcd71cf288-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acf447f1-dd36-4897-9a3d-9fdcd71cf288-istio-data\") pod \"acf447f1-dd36-4897-9a3d-9fdcd71cf288\" (UID: \"acf447f1-dd36-4897-9a3d-9fdcd71cf288\") "
Nov 04 03:44:51.383700 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.383668    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/acf447f1-dd36-4897-9a3d-9fdcd71cf288-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acf447f1-dd36-4897-9a3d-9fdcd71cf288-workload-certs\") pod \"acf447f1-dd36-4897-9a3d-9fdcd71cf288\" (UID: \"acf447f1-dd36-4897-9a3d-9fdcd71cf288\") "
Nov 04 03:44:51.383700 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.383686    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/acf447f1-dd36-4897-9a3d-9fdcd71cf288-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acf447f1-dd36-4897-9a3d-9fdcd71cf288-istio-envoy\") pod \"acf447f1-dd36-4897-9a3d-9fdcd71cf288\" (UID: \"acf447f1-dd36-4897-9a3d-9fdcd71cf288\") "
Nov 04 03:44:51.384276 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.383730    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/acf447f1-dd36-4897-9a3d-9fdcd71cf288-kube-api-access-wxwfk\" (UniqueName: \"kubernetes.io/projected/acf447f1-dd36-4897-9a3d-9fdcd71cf288-kube-api-access-wxwfk\") pod \"acf447f1-dd36-4897-9a3d-9fdcd71cf288\" (UID: \"acf447f1-dd36-4897-9a3d-9fdcd71cf288\") "
Nov 04 03:44:51.384276 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.383748    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/acf447f1-dd36-4897-9a3d-9fdcd71cf288-istio-token\" (UniqueName: \"kubernetes.io/projected/acf447f1-dd36-4897-9a3d-9fdcd71cf288-istio-token\") pod \"acf447f1-dd36-4897-9a3d-9fdcd71cf288\" (UID: \"acf447f1-dd36-4897-9a3d-9fdcd71cf288\") "
Nov 04 03:44:51.384276 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.383785    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/acf447f1-dd36-4897-9a3d-9fdcd71cf288-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf447f1-dd36-4897-9a3d-9fdcd71cf288-workload-socket\") pod \"acf447f1-dd36-4897-9a3d-9fdcd71cf288\" (UID: \"acf447f1-dd36-4897-9a3d-9fdcd71cf288\") "
Nov 04 03:44:51.384276 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.383801    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/acf447f1-dd36-4897-9a3d-9fdcd71cf288-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf447f1-dd36-4897-9a3d-9fdcd71cf288-credential-socket\") pod \"acf447f1-dd36-4897-9a3d-9fdcd71cf288\" (UID: \"acf447f1-dd36-4897-9a3d-9fdcd71cf288\") "
Nov 04 03:44:51.384276 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.383837    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/acf447f1-dd36-4897-9a3d-9fdcd71cf288-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acf447f1-dd36-4897-9a3d-9fdcd71cf288-istiod-ca-cert\") pod \"acf447f1-dd36-4897-9a3d-9fdcd71cf288\" (UID: \"acf447f1-dd36-4897-9a3d-9fdcd71cf288\") "
Nov 04 03:44:51.384276 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.383856    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/acf447f1-dd36-4897-9a3d-9fdcd71cf288-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acf447f1-dd36-4897-9a3d-9fdcd71cf288-istio-podinfo\") pod \"acf447f1-dd36-4897-9a3d-9fdcd71cf288\" (UID: \"acf447f1-dd36-4897-9a3d-9fdcd71cf288\") "
Nov 04 03:44:51.384276 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.383912    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acf447f1-dd36-4897-9a3d-9fdcd71cf288-istio-data" pod "acf447f1-dd36-4897-9a3d-9fdcd71cf288" (UID: "acf447f1-dd36-4897-9a3d-9fdcd71cf288"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:51.384276 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.384157    5609 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/66120169-8cf1-4166-93b9-2ae671872165-workload-socket\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:51.384276 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.384168    5609 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/66120169-8cf1-4166-93b9-2ae671872165-istio-data\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:51.384276 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.384177    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jphnr\" (UniqueName: \"kubernetes.io/projected/66120169-8cf1-4166-93b9-2ae671872165-kube-api-access-jphnr\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:51.384276 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.384186    5609 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/66120169-8cf1-4166-93b9-2ae671872165-istio-envoy\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:51.384276 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.384194    5609 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/66120169-8cf1-4166-93b9-2ae671872165-credential-socket\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:51.384276 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.384202    5609 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/66120169-8cf1-4166-93b9-2ae671872165-istio-token\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:51.384276 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.384210    5609 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acf447f1-dd36-4897-9a3d-9fdcd71cf288-istio-data\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:51.384276 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.384218    5609 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/66120169-8cf1-4166-93b9-2ae671872165-istio-podinfo\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:51.385382 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.385351    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acf447f1-dd36-4897-9a3d-9fdcd71cf288-workload-socket" pod "acf447f1-dd36-4897-9a3d-9fdcd71cf288" (UID: "acf447f1-dd36-4897-9a3d-9fdcd71cf288"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:51.385539 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.385494    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acf447f1-dd36-4897-9a3d-9fdcd71cf288-credential-socket" pod "acf447f1-dd36-4897-9a3d-9fdcd71cf288" (UID: "acf447f1-dd36-4897-9a3d-9fdcd71cf288"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:51.385671 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.385645    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/acf447f1-dd36-4897-9a3d-9fdcd71cf288-istiod-ca-cert" pod "acf447f1-dd36-4897-9a3d-9fdcd71cf288" (UID: "acf447f1-dd36-4897-9a3d-9fdcd71cf288"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:44:51.385791 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.385710    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acf447f1-dd36-4897-9a3d-9fdcd71cf288-workload-certs" pod "acf447f1-dd36-4897-9a3d-9fdcd71cf288" (UID: "acf447f1-dd36-4897-9a3d-9fdcd71cf288"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:51.386533 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.386498    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acf447f1-dd36-4897-9a3d-9fdcd71cf288-istio-token" pod "acf447f1-dd36-4897-9a3d-9fdcd71cf288" (UID: "acf447f1-dd36-4897-9a3d-9fdcd71cf288"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:44:51.386665 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.386641    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acf447f1-dd36-4897-9a3d-9fdcd71cf288-istio-envoy" pod "acf447f1-dd36-4897-9a3d-9fdcd71cf288" (UID: "acf447f1-dd36-4897-9a3d-9fdcd71cf288"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:51.387694 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.387670    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/acf447f1-dd36-4897-9a3d-9fdcd71cf288-istio-podinfo" pod "acf447f1-dd36-4897-9a3d-9fdcd71cf288" (UID: "acf447f1-dd36-4897-9a3d-9fdcd71cf288"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:44:51.388053 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.388023    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acf447f1-dd36-4897-9a3d-9fdcd71cf288-kube-api-access-wxwfk" pod "acf447f1-dd36-4897-9a3d-9fdcd71cf288" (UID: "acf447f1-dd36-4897-9a3d-9fdcd71cf288"). InnerVolumeSpecName "kube-api-access-wxwfk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:44:51.484792 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.484755    5609 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acf447f1-dd36-4897-9a3d-9fdcd71cf288-istiod-ca-cert\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:51.484792 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.484788    5609 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acf447f1-dd36-4897-9a3d-9fdcd71cf288-istio-podinfo\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:51.484792 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.484797    5609 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acf447f1-dd36-4897-9a3d-9fdcd71cf288-workload-certs\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:51.484792 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.484807    5609 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acf447f1-dd36-4897-9a3d-9fdcd71cf288-istio-envoy\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:51.485051 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.484819    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wxwfk\" (UniqueName: \"kubernetes.io/projected/acf447f1-dd36-4897-9a3d-9fdcd71cf288-kube-api-access-wxwfk\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:51.485051 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.484828    5609 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acf447f1-dd36-4897-9a3d-9fdcd71cf288-istio-token\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:51.485051 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.484838    5609 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf447f1-dd36-4897-9a3d-9fdcd71cf288-workload-socket\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:51.485051 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:51.484847    5609 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf447f1-dd36-4897-9a3d-9fdcd71cf288-credential-socket\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:52.000865 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:52.000833    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x"
Nov 04 03:44:52.000865 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:52.000851    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x" event={"ID":"66120169-8cf1-4166-93b9-2ae671872165","Type":"ContainerDied","Data":"45c4c35cb9a9bbf1a798a0a61f89fd582d52c37763814ab1ba7ebc660008f394"}
Nov 04 03:44:52.001119 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:52.000917    5609 scope.go:122] "RemoveContainer" containerID="2cad8d652e639b0c844641145cb8345e20739565d7838e050b2a11e4584e2313"
Nov 04 03:44:52.002634 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:52.002608    5609 generic.go:356] "Generic (PLEG): container finished" podID="acf447f1-dd36-4897-9a3d-9fdcd71cf288" containerID="7434e10cb1473596a213e602e936076b778c999202b4f560b994dc26082981b4" exitCode=0
Nov 04 03:44:52.002775 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:52.002677    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng"
Nov 04 03:44:52.002775 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:52.002681    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng" event={"ID":"acf447f1-dd36-4897-9a3d-9fdcd71cf288","Type":"ContainerDied","Data":"7434e10cb1473596a213e602e936076b778c999202b4f560b994dc26082981b4"}
Nov 04 03:44:52.002775 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:52.002720    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng" event={"ID":"acf447f1-dd36-4897-9a3d-9fdcd71cf288","Type":"ContainerDied","Data":"41f773ded09a869acecc039e92d92d44d89da2ca340aec573af58e5e13a7585d"}
Nov 04 03:44:52.017144 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:52.017125    5609 scope.go:122] "RemoveContainer" containerID="7434e10cb1473596a213e602e936076b778c999202b4f560b994dc26082981b4"
Nov 04 03:44:52.026813 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:52.026781    5609 scope.go:122] "RemoveContainer" containerID="7434e10cb1473596a213e602e936076b778c999202b4f560b994dc26082981b4"
Nov 04 03:44:52.027105 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:44:52.027082    5609 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7434e10cb1473596a213e602e936076b778c999202b4f560b994dc26082981b4\": container with ID starting with 7434e10cb1473596a213e602e936076b778c999202b4f560b994dc26082981b4 not found: ID does not exist" containerID="7434e10cb1473596a213e602e936076b778c999202b4f560b994dc26082981b4"
Nov 04 03:44:52.027173 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:52.027114    5609 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7434e10cb1473596a213e602e936076b778c999202b4f560b994dc26082981b4"} err="failed to get container status \"7434e10cb1473596a213e602e936076b778c999202b4f560b994dc26082981b4\": rpc error: code = NotFound desc = could not find container \"7434e10cb1473596a213e602e936076b778c999202b4f560b994dc26082981b4\": container with ID starting with 7434e10cb1473596a213e602e936076b778c999202b4f560b994dc26082981b4 not found: ID does not exist"
Nov 04 03:44:52.223050 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:52.223013    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x"]
Nov 04 03:44:52.227966 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:52.227938    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-x4d9x"]
Nov 04 03:44:52.331364 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:52.331333    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng"]
Nov 04 03:44:52.335237 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:52.335208    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-pjdng"]
Nov 04 03:44:52.543101 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:52.543025    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw" podUID="16fc6217-3711-4d08-ad50-540132d280ad" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:44:53.534366 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:53.534332    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="66120169-8cf1-4166-93b9-2ae671872165" path="/var/lib/kubelet/pods/66120169-8cf1-4166-93b9-2ae671872165/volumes"
Nov 04 03:44:53.534796 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:44:53.534782    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="acf447f1-dd36-4897-9a3d-9fdcd71cf288" path="/var/lib/kubelet/pods/acf447f1-dd36-4897-9a3d-9fdcd71cf288/volumes"
Nov 04 03:44:57.351676 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:44:57.351645    5609 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-8b97550813bf: secret "agc-cred-8b97550813bf" not found
Nov 04 03:44:57.351676 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:44:57.351672    5609 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-df4cf9975-hk75w: secret "agc-cred-8b97550813bf" not found
Nov 04 03:44:57.352095 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:44:57.351726    5609 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d-provider-credentials podName:0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d nodeName:}" failed. No retries permitted until 2026-11-04 03:45:13.351711749 +0000 UTC m=+1655.928364654 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d-provider-credentials") pod "payload-processing-external-model-df4cf9975-hk75w" (UID: "0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d") : secret "agc-cred-8b97550813bf" not found
Nov 04 03:45:00.175082 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:00.175045    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6d170d-29896065-j22ll"]
Nov 04 03:45:00.175780 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:00.175567    5609 cpu_manager.go:414] "containerMap: removing container" podUID="66120169-8cf1-4166-93b9-2ae671872165" containerName="istio-proxy"
Nov 04 03:45:00.175780 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:00.175582    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="66120169-8cf1-4166-93b9-2ae671872165" containerName="istio-proxy"
Nov 04 03:45:00.175780 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:00.175593    5609 cpu_manager.go:414] "containerMap: removing container" podUID="66120169-8cf1-4166-93b9-2ae671872165" containerName="istio-proxy"
Nov 04 03:45:00.175780 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:00.175600    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="66120169-8cf1-4166-93b9-2ae671872165" containerName="istio-proxy"
Nov 04 03:45:00.175780 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:00.175616    5609 cpu_manager.go:414] "containerMap: removing container" podUID="cf09732a-b411-43d6-81dc-0240f879a783" containerName="payload-processing"
Nov 04 03:45:00.175780 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:00.175622    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf09732a-b411-43d6-81dc-0240f879a783" containerName="payload-processing"
Nov 04 03:45:00.175780 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:00.175671    5609 cpu_manager.go:414] "containerMap: removing container" podUID="acf447f1-dd36-4897-9a3d-9fdcd71cf288" containerName="istio-proxy"
Nov 04 03:45:00.175780 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:00.175679    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="acf447f1-dd36-4897-9a3d-9fdcd71cf288" containerName="istio-proxy"
Nov 04 03:45:00.175780 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:00.175692    5609 cpu_manager.go:414] "containerMap: removing container" podUID="2e3ba585-0eb0-44bb-955c-3b07f75123f9" containerName="payload-pre-processing"
Nov 04 03:45:00.175780 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:00.175700    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2e3ba585-0eb0-44bb-955c-3b07f75123f9" containerName="payload-pre-processing"
Nov 04 03:45:00.175780 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:00.175774    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="acf447f1-dd36-4897-9a3d-9fdcd71cf288" containerName="istio-proxy"
Nov 04 03:45:00.175780 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:00.175781    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="2e3ba585-0eb0-44bb-955c-3b07f75123f9" containerName="payload-pre-processing"
Nov 04 03:45:00.175780 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:00.175790    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="66120169-8cf1-4166-93b9-2ae671872165" containerName="istio-proxy"
Nov 04 03:45:00.176429 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:00.175797    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="66120169-8cf1-4166-93b9-2ae671872165" containerName="istio-proxy"
Nov 04 03:45:00.176429 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:00.175805    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="cf09732a-b411-43d6-81dc-0240f879a783" containerName="payload-processing"
Nov 04 03:45:00.178655 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:00.178635    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29896065-fzlxb"]
Nov 04 03:45:00.178794 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:00.178770    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6d170d-29896065-j22ll"
Nov 04 03:45:00.182070 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:00.182049    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896065-fzlxb"
Nov 04 03:45:00.229183 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:00.229148    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x9cnx\""
Nov 04 03:45:00.270386 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:00.270348    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29896065-fzlxb"]
Nov 04 03:45:00.281106 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:00.281075    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6d170d-29896065-j22ll"]
Nov 04 03:45:00.284080 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:00.284039    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0adb4153-ff09-4f78-b28b-d976700f145f-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-6d170d-29896065-j22ll\" (UID: \"0adb4153-ff09-4f78-b28b-d976700f145f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6d170d-29896065-j22ll"
Nov 04 03:45:00.284192 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:00.284101    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twhd8\" (UniqueName: \"kubernetes.io/projected/97ee046b-2ad6-44b4-9e1f-7286d8988b2c-kube-api-access-twhd8\") pod \"maas-api-key-cleanup-29896065-fzlxb\" (UID: \"97ee046b-2ad6-44b4-9e1f-7286d8988b2c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896065-fzlxb"
Nov 04 03:45:00.284247 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:00.284224    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97ee046b-2ad6-44b4-9e1f-7286d8988b2c-maas-api-service-ca\") pod \"maas-api-key-cleanup-29896065-fzlxb\" (UID: \"97ee046b-2ad6-44b4-9e1f-7286d8988b2c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896065-fzlxb"
Nov 04 03:45:00.284283 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:00.284272    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmlls\" (UniqueName: \"kubernetes.io/projected/0adb4153-ff09-4f78-b28b-d976700f145f-kube-api-access-pmlls\") pod \"maas-api-key-cleanup-e2e-extmod-a-6d170d-29896065-j22ll\" (UID: \"0adb4153-ff09-4f78-b28b-d976700f145f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6d170d-29896065-j22ll"
Nov 04 03:45:00.385261 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:00.385223    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmlls\" (UniqueName: \"kubernetes.io/projected/0adb4153-ff09-4f78-b28b-d976700f145f-kube-api-access-pmlls\") pod \"maas-api-key-cleanup-e2e-extmod-a-6d170d-29896065-j22ll\" (UID: \"0adb4153-ff09-4f78-b28b-d976700f145f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6d170d-29896065-j22ll"
Nov 04 03:45:00.385444 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:00.385331    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0adb4153-ff09-4f78-b28b-d976700f145f-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-6d170d-29896065-j22ll\" (UID: \"0adb4153-ff09-4f78-b28b-d976700f145f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6d170d-29896065-j22ll"
Nov 04 03:45:00.385444 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:00.385355    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twhd8\" (UniqueName: \"kubernetes.io/projected/97ee046b-2ad6-44b4-9e1f-7286d8988b2c-kube-api-access-twhd8\") pod \"maas-api-key-cleanup-29896065-fzlxb\" (UID: \"97ee046b-2ad6-44b4-9e1f-7286d8988b2c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896065-fzlxb"
Nov 04 03:45:00.385444 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:00.385418    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97ee046b-2ad6-44b4-9e1f-7286d8988b2c-maas-api-service-ca\") pod \"maas-api-key-cleanup-29896065-fzlxb\" (UID: \"97ee046b-2ad6-44b4-9e1f-7286d8988b2c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896065-fzlxb"
Nov 04 03:45:00.386073 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:00.386053    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97ee046b-2ad6-44b4-9e1f-7286d8988b2c-maas-api-service-ca\") pod \"maas-api-key-cleanup-29896065-fzlxb\" (UID: \"97ee046b-2ad6-44b4-9e1f-7286d8988b2c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896065-fzlxb"
Nov 04 03:45:00.386158 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:00.386135    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0adb4153-ff09-4f78-b28b-d976700f145f-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-6d170d-29896065-j22ll\" (UID: \"0adb4153-ff09-4f78-b28b-d976700f145f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6d170d-29896065-j22ll"
Nov 04 03:45:00.456610 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:00.456575    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmlls\" (UniqueName: \"kubernetes.io/projected/0adb4153-ff09-4f78-b28b-d976700f145f-kube-api-access-pmlls\") pod \"maas-api-key-cleanup-e2e-extmod-a-6d170d-29896065-j22ll\" (UID: \"0adb4153-ff09-4f78-b28b-d976700f145f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6d170d-29896065-j22ll"
Nov 04 03:45:00.456610 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:00.456592    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-twhd8\" (UniqueName: \"kubernetes.io/projected/97ee046b-2ad6-44b4-9e1f-7286d8988b2c-kube-api-access-twhd8\") pod \"maas-api-key-cleanup-29896065-fzlxb\" (UID: \"97ee046b-2ad6-44b4-9e1f-7286d8988b2c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896065-fzlxb"
Nov 04 03:45:00.494993 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:00.494955    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6d170d-29896065-j22ll"
Nov 04 03:45:00.502332 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:00.502304    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896065-fzlxb"
Nov 04 03:45:00.877319 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:00.877244    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6d170d-29896065-j22ll"]
Nov 04 03:45:00.882206 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:45:00.882177    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0adb4153_ff09_4f78_b28b_d976700f145f.slice/crio-11ca1665afe3b1af923162a043bd204a942d62d28d4dbf627033c37f52fcecda WatchSource:0}: Error finding container 11ca1665afe3b1af923162a043bd204a942d62d28d4dbf627033c37f52fcecda: Status 404 returned error can't find the container with id 11ca1665afe3b1af923162a043bd204a942d62d28d4dbf627033c37f52fcecda
Nov 04 03:45:00.929211 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:00.929177    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29896065-fzlxb"]
Nov 04 03:45:00.933342 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:45:00.933312    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97ee046b_2ad6_44b4_9e1f_7286d8988b2c.slice/crio-61955c7461538387e215a514cceb814a5354233f6c5aaf733363a89e4e050f7b WatchSource:0}: Error finding container 61955c7461538387e215a514cceb814a5354233f6c5aaf733363a89e4e050f7b: Status 404 returned error can't find the container with id 61955c7461538387e215a514cceb814a5354233f6c5aaf733363a89e4e050f7b
Nov 04 03:45:01.066443 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:01.066381    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896065-fzlxb" event={"ID":"97ee046b-2ad6-44b4-9e1f-7286d8988b2c","Type":"ContainerStarted","Data":"61955c7461538387e215a514cceb814a5354233f6c5aaf733363a89e4e050f7b"}
Nov 04 03:45:01.067743 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:01.067717    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6d170d-29896065-j22ll" event={"ID":"0adb4153-ff09-4f78-b28b-d976700f145f","Type":"ContainerStarted","Data":"11ca1665afe3b1af923162a043bd204a942d62d28d4dbf627033c37f52fcecda"}
Nov 04 03:45:02.076079 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:02.076047    5609 generic.go:356] "Generic (PLEG): container finished" podID="97ee046b-2ad6-44b4-9e1f-7286d8988b2c" containerID="a93b206ac9c4fb97ad8a108795806436a12dad3685f4ccf70c3d1bee22d7cb16" exitCode=60
Nov 04 03:45:02.076552 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:02.076131    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896065-fzlxb" event={"ID":"97ee046b-2ad6-44b4-9e1f-7286d8988b2c","Type":"ContainerDied","Data":"a93b206ac9c4fb97ad8a108795806436a12dad3685f4ccf70c3d1bee22d7cb16"}
Nov 04 03:45:02.076552 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:02.076360    5609 scope.go:122] "RemoveContainer" containerID="a93b206ac9c4fb97ad8a108795806436a12dad3685f4ccf70c3d1bee22d7cb16"
Nov 04 03:45:02.078169 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:02.078148    5609 generic.go:356] "Generic (PLEG): container finished" podID="0adb4153-ff09-4f78-b28b-d976700f145f" containerID="7609452b1e7e98a3725e64e5b5229cdb82bc5712130d1820d39c143b68b53f79" exitCode=60
Nov 04 03:45:02.078280 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:02.078174    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6d170d-29896065-j22ll" event={"ID":"0adb4153-ff09-4f78-b28b-d976700f145f","Type":"ContainerDied","Data":"7609452b1e7e98a3725e64e5b5229cdb82bc5712130d1820d39c143b68b53f79"}
Nov 04 03:45:02.078484 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:02.078467    5609 scope.go:122] "RemoveContainer" containerID="7609452b1e7e98a3725e64e5b5229cdb82bc5712130d1820d39c143b68b53f79"
Nov 04 03:45:02.544275 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:02.544185    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw" podUID="16fc6217-3711-4d08-ad50-540132d280ad" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:45:03.087018 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:03.086986    5609 generic.go:356] "Generic (PLEG): container finished" podID="97ee046b-2ad6-44b4-9e1f-7286d8988b2c" containerID="e6e26d03e85371a9640cd1a668b54ef7239776577925f9da03d7513220da6238" exitCode=60
Nov 04 03:45:03.087537 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:03.087071    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896065-fzlxb" event={"ID":"97ee046b-2ad6-44b4-9e1f-7286d8988b2c","Type":"ContainerDied","Data":"e6e26d03e85371a9640cd1a668b54ef7239776577925f9da03d7513220da6238"}
Nov 04 03:45:03.087537 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:03.087135    5609 scope.go:122] "RemoveContainer" containerID="a93b206ac9c4fb97ad8a108795806436a12dad3685f4ccf70c3d1bee22d7cb16"
Nov 04 03:45:03.087537 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:03.087334    5609 scope.go:122] "RemoveContainer" containerID="e6e26d03e85371a9640cd1a668b54ef7239776577925f9da03d7513220da6238"
Nov 04 03:45:03.087668 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:45:03.087617    5609 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29896065-fzlxb_odh-ai-gateway-infra(97ee046b-2ad6-44b4-9e1f-7286d8988b2c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896065-fzlxb" podUID="97ee046b-2ad6-44b4-9e1f-7286d8988b2c"
Nov 04 03:45:03.089303 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:03.089279    5609 generic.go:356] "Generic (PLEG): container finished" podID="0adb4153-ff09-4f78-b28b-d976700f145f" containerID="58baa9ba47f12af4dacf0a7e9c39e6e3c46aa4a2cca88421a6eb4792f8067f29" exitCode=60
Nov 04 03:45:03.089503 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:03.089371    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6d170d-29896065-j22ll" event={"ID":"0adb4153-ff09-4f78-b28b-d976700f145f","Type":"ContainerDied","Data":"58baa9ba47f12af4dacf0a7e9c39e6e3c46aa4a2cca88421a6eb4792f8067f29"}
Nov 04 03:45:03.089627 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:03.089613    5609 scope.go:122] "RemoveContainer" containerID="58baa9ba47f12af4dacf0a7e9c39e6e3c46aa4a2cca88421a6eb4792f8067f29"
Nov 04 03:45:03.089825 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:45:03.089810    5609 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-6d170d-29896065-j22ll_odh-ai-gateway-infra(0adb4153-ff09-4f78-b28b-d976700f145f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6d170d-29896065-j22ll" podUID="0adb4153-ff09-4f78-b28b-d976700f145f"
Nov 04 03:45:03.099741 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:03.099724    5609 scope.go:122] "RemoveContainer" containerID="7609452b1e7e98a3725e64e5b5229cdb82bc5712130d1820d39c143b68b53f79"
Nov 04 03:45:04.097303 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:04.097274    5609 scope.go:122] "RemoveContainer" containerID="e6e26d03e85371a9640cd1a668b54ef7239776577925f9da03d7513220da6238"
Nov 04 03:45:04.097787 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:45:04.097600    5609 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29896065-fzlxb_odh-ai-gateway-infra(97ee046b-2ad6-44b4-9e1f-7286d8988b2c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896065-fzlxb" podUID="97ee046b-2ad6-44b4-9e1f-7286d8988b2c"
Nov 04 03:45:04.099022 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:04.099004    5609 scope.go:122] "RemoveContainer" containerID="58baa9ba47f12af4dacf0a7e9c39e6e3c46aa4a2cca88421a6eb4792f8067f29"
Nov 04 03:45:04.099210 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:45:04.099193    5609 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-6d170d-29896065-j22ll_odh-ai-gateway-infra(0adb4153-ff09-4f78-b28b-d976700f145f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6d170d-29896065-j22ll" podUID="0adb4153-ff09-4f78-b28b-d976700f145f"
Nov 04 03:45:12.544112 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:12.544074    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw" podUID="16fc6217-3711-4d08-ad50-540132d280ad" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:45:13.435041 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:45:13.435001    5609 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-8b97550813bf: secret "agc-cred-8b97550813bf" not found
Nov 04 03:45:13.435041 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:45:13.435033    5609 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-df4cf9975-hk75w: secret "agc-cred-8b97550813bf" not found
Nov 04 03:45:13.435265 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:45:13.435103    5609 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d-provider-credentials podName:0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d nodeName:}" failed. No retries permitted until 2026-11-04 03:45:45.435089865 +0000 UTC m=+1688.011742757 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d-provider-credentials") pod "payload-processing-external-model-df4cf9975-hk75w" (UID: "0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d") : secret "agc-cred-8b97550813bf" not found
Nov 04 03:45:14.528255 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:14.528226    5609 scope.go:122] "RemoveContainer" containerID="e6e26d03e85371a9640cd1a668b54ef7239776577925f9da03d7513220da6238"
Nov 04 03:45:14.528664 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:14.528313    5609 scope.go:122] "RemoveContainer" containerID="58baa9ba47f12af4dacf0a7e9c39e6e3c46aa4a2cca88421a6eb4792f8067f29"
Nov 04 03:45:15.172816 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:15.172720    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896065-fzlxb" event={"ID":"97ee046b-2ad6-44b4-9e1f-7286d8988b2c","Type":"ContainerStarted","Data":"4d368fbf4ba31128544e0546c5d87e27821447a0f3eb5460f4e91b265bc5b1d8"}
Nov 04 03:45:15.174695 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:15.174670    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6d170d-29896065-j22ll" event={"ID":"0adb4153-ff09-4f78-b28b-d976700f145f","Type":"ContainerStarted","Data":"04c1a6911e431963aa61e7d19017647840efd01e9c2aa0e24ba1c14e4739d181"}
Nov 04 03:45:15.283530 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:15.283467    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896065-fzlxb" podStartSLOduration=15.283450132 podStartE2EDuration="15.283450132s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:45:15.282207575 +0000 UTC m=+1657.858860528" watchObservedRunningTime="2026-11-04 03:45:15.283450132 +0000 UTC m=+1657.860103047"
Nov 04 03:45:15.394810 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:15.394761    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6d170d-29896065-j22ll" podStartSLOduration=15.394745516 podStartE2EDuration="15.394745516s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:45:15.391756584 +0000 UTC m=+1657.968409497" watchObservedRunningTime="2026-11-04 03:45:15.394745516 +0000 UTC m=+1657.971398429"
Nov 04 03:45:16.183880 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:16.183850    5609 generic.go:356] "Generic (PLEG): container finished" podID="0adb4153-ff09-4f78-b28b-d976700f145f" containerID="04c1a6911e431963aa61e7d19017647840efd01e9c2aa0e24ba1c14e4739d181" exitCode=60
Nov 04 03:45:16.184341 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:16.183922    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6d170d-29896065-j22ll" event={"ID":"0adb4153-ff09-4f78-b28b-d976700f145f","Type":"ContainerDied","Data":"04c1a6911e431963aa61e7d19017647840efd01e9c2aa0e24ba1c14e4739d181"}
Nov 04 03:45:16.184341 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:16.183981    5609 scope.go:122] "RemoveContainer" containerID="58baa9ba47f12af4dacf0a7e9c39e6e3c46aa4a2cca88421a6eb4792f8067f29"
Nov 04 03:45:16.184341 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:16.184292    5609 scope.go:122] "RemoveContainer" containerID="04c1a6911e431963aa61e7d19017647840efd01e9c2aa0e24ba1c14e4739d181"
Nov 04 03:45:16.184606 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:45:16.184583    5609 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-6d170d-29896065-j22ll_odh-ai-gateway-infra(0adb4153-ff09-4f78-b28b-d976700f145f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6d170d-29896065-j22ll" podUID="0adb4153-ff09-4f78-b28b-d976700f145f"
Nov 04 03:45:16.186079 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:16.186062    5609 generic.go:356] "Generic (PLEG): container finished" podID="97ee046b-2ad6-44b4-9e1f-7286d8988b2c" containerID="4d368fbf4ba31128544e0546c5d87e27821447a0f3eb5460f4e91b265bc5b1d8" exitCode=60
Nov 04 03:45:16.186191 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:16.186085    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896065-fzlxb" event={"ID":"97ee046b-2ad6-44b4-9e1f-7286d8988b2c","Type":"ContainerDied","Data":"4d368fbf4ba31128544e0546c5d87e27821447a0f3eb5460f4e91b265bc5b1d8"}
Nov 04 03:45:16.186390 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:16.186374    5609 scope.go:122] "RemoveContainer" containerID="4d368fbf4ba31128544e0546c5d87e27821447a0f3eb5460f4e91b265bc5b1d8"
Nov 04 03:45:16.186647 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:45:16.186622    5609 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29896065-fzlxb_odh-ai-gateway-infra(97ee046b-2ad6-44b4-9e1f-7286d8988b2c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896065-fzlxb" podUID="97ee046b-2ad6-44b4-9e1f-7286d8988b2c"
Nov 04 03:45:16.195927 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:16.195910    5609 scope.go:122] "RemoveContainer" containerID="e6e26d03e85371a9640cd1a668b54ef7239776577925f9da03d7513220da6238"
Nov 04 03:45:16.293944 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:16.293914    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29896065-fzlxb"]
Nov 04 03:45:16.404940 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:16.404905    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6d170d-29896065-j22ll"]
Nov 04 03:45:17.346634 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:17.346612    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896065-fzlxb"
Nov 04 03:45:17.370525 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:17.370503    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6d170d-29896065-j22ll"
Nov 04 03:45:17.474908 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:17.474821    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0adb4153-ff09-4f78-b28b-d976700f145f-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0adb4153-ff09-4f78-b28b-d976700f145f-maas-api-service-ca\") pod \"0adb4153-ff09-4f78-b28b-d976700f145f\" (UID: \"0adb4153-ff09-4f78-b28b-d976700f145f\") "
Nov 04 03:45:17.474908 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:17.474873    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0adb4153-ff09-4f78-b28b-d976700f145f-kube-api-access-pmlls\" (UniqueName: \"kubernetes.io/projected/0adb4153-ff09-4f78-b28b-d976700f145f-kube-api-access-pmlls\") pod \"0adb4153-ff09-4f78-b28b-d976700f145f\" (UID: \"0adb4153-ff09-4f78-b28b-d976700f145f\") "
Nov 04 03:45:17.475146 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:17.474931    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/97ee046b-2ad6-44b4-9e1f-7286d8988b2c-kube-api-access-twhd8\" (UniqueName: \"kubernetes.io/projected/97ee046b-2ad6-44b4-9e1f-7286d8988b2c-kube-api-access-twhd8\") pod \"97ee046b-2ad6-44b4-9e1f-7286d8988b2c\" (UID: \"97ee046b-2ad6-44b4-9e1f-7286d8988b2c\") "
Nov 04 03:45:17.475146 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:17.474989    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/97ee046b-2ad6-44b4-9e1f-7286d8988b2c-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97ee046b-2ad6-44b4-9e1f-7286d8988b2c-maas-api-service-ca\") pod \"97ee046b-2ad6-44b4-9e1f-7286d8988b2c\" (UID: \"97ee046b-2ad6-44b4-9e1f-7286d8988b2c\") "
Nov 04 03:45:17.475260 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:17.475195    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0adb4153-ff09-4f78-b28b-d976700f145f-maas-api-service-ca" pod "0adb4153-ff09-4f78-b28b-d976700f145f" (UID: "0adb4153-ff09-4f78-b28b-d976700f145f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:45:17.475389 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:17.475364    5609 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0adb4153-ff09-4f78-b28b-d976700f145f-maas-api-service-ca\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:45:17.475543 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:17.475436    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97ee046b-2ad6-44b4-9e1f-7286d8988b2c-maas-api-service-ca" pod "97ee046b-2ad6-44b4-9e1f-7286d8988b2c" (UID: "97ee046b-2ad6-44b4-9e1f-7286d8988b2c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:45:17.477107 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:17.477084    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0adb4153-ff09-4f78-b28b-d976700f145f-kube-api-access-pmlls" pod "0adb4153-ff09-4f78-b28b-d976700f145f" (UID: "0adb4153-ff09-4f78-b28b-d976700f145f"). InnerVolumeSpecName "kube-api-access-pmlls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:45:17.477203 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:17.477121    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97ee046b-2ad6-44b4-9e1f-7286d8988b2c-kube-api-access-twhd8" pod "97ee046b-2ad6-44b4-9e1f-7286d8988b2c" (UID: "97ee046b-2ad6-44b4-9e1f-7286d8988b2c"). InnerVolumeSpecName "kube-api-access-twhd8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:45:17.576450 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:17.576413    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pmlls\" (UniqueName: \"kubernetes.io/projected/0adb4153-ff09-4f78-b28b-d976700f145f-kube-api-access-pmlls\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:45:17.576450 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:17.576450    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-twhd8\" (UniqueName: \"kubernetes.io/projected/97ee046b-2ad6-44b4-9e1f-7286d8988b2c-kube-api-access-twhd8\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:45:17.576662 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:17.576466    5609 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97ee046b-2ad6-44b4-9e1f-7286d8988b2c-maas-api-service-ca\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:45:18.203107 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:18.203080    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896065-fzlxb"
Nov 04 03:45:18.203274 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:18.203127    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896065-fzlxb" event={"ID":"97ee046b-2ad6-44b4-9e1f-7286d8988b2c","Type":"ContainerDied","Data":"61955c7461538387e215a514cceb814a5354233f6c5aaf733363a89e4e050f7b"}
Nov 04 03:45:18.203274 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:18.203190    5609 scope.go:122] "RemoveContainer" containerID="4d368fbf4ba31128544e0546c5d87e27821447a0f3eb5460f4e91b265bc5b1d8"
Nov 04 03:45:18.204659 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:18.204636    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6d170d-29896065-j22ll" event={"ID":"0adb4153-ff09-4f78-b28b-d976700f145f","Type":"ContainerDied","Data":"11ca1665afe3b1af923162a043bd204a942d62d28d4dbf627033c37f52fcecda"}
Nov 04 03:45:18.204741 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:18.204709    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6d170d-29896065-j22ll"
Nov 04 03:45:18.216302 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:18.216280    5609 scope.go:122] "RemoveContainer" containerID="04c1a6911e431963aa61e7d19017647840efd01e9c2aa0e24ba1c14e4739d181"
Nov 04 03:45:18.363427 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:18.363386    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6d170d-29896065-j22ll"]
Nov 04 03:45:18.410028 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:18.409993    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6d170d-29896065-j22ll"]
Nov 04 03:45:18.476979 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:18.476893    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29896065-fzlxb"]
Nov 04 03:45:18.479619 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:18.479595    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29896065-fzlxb"]
Nov 04 03:45:18.559175 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:45:18.559139    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:45:19.534559 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:19.534525    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0adb4153-ff09-4f78-b28b-d976700f145f" path="/var/lib/kubelet/pods/0adb4153-ff09-4f78-b28b-d976700f145f/volumes"
Nov 04 03:45:19.535391 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:19.535137    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="97ee046b-2ad6-44b4-9e1f-7286d8988b2c" path="/var/lib/kubelet/pods/97ee046b-2ad6-44b4-9e1f-7286d8988b2c/volumes"
Nov 04 03:45:21.535181 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:21.535147    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw" podUID="16fc6217-3711-4d08-ad50-540132d280ad" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:45:31.535298 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:31.535264    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw" podUID="16fc6217-3711-4d08-ad50-540132d280ad" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:45:41.535538 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:41.535503    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw" podUID="16fc6217-3711-4d08-ad50-540132d280ad" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:45:45.451441 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:45:45.451383    5609 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-8b97550813bf: secret "agc-cred-8b97550813bf" not found
Nov 04 03:45:45.451441 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:45:45.451432    5609 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-df4cf9975-hk75w: secret "agc-cred-8b97550813bf" not found
Nov 04 03:45:45.451904 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:45:45.451492    5609 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d-provider-credentials podName:0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d nodeName:}" failed. No retries permitted until 2026-11-04 03:46:49.45147877 +0000 UTC m=+1752.028131663 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d-provider-credentials") pod "payload-processing-external-model-df4cf9975-hk75w" (UID: "0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d") : secret "agc-cred-8b97550813bf" not found
Nov 04 03:45:48.559634 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:45:48.559595    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:45:51.535680 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:45:51.535644    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw" podUID="16fc6217-3711-4d08-ad50-540132d280ad" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:46:01.535461 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:46:01.535422    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw" podUID="16fc6217-3711-4d08-ad50-540132d280ad" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:46:11.535984 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:46:11.535947    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw" podUID="16fc6217-3711-4d08-ad50-540132d280ad" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:46:18.560014 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:46:18.559936    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:46:21.535800 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:46:21.535765    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw" podUID="16fc6217-3711-4d08-ad50-540132d280ad" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:46:31.535238 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:46:31.535201    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw" podUID="16fc6217-3711-4d08-ad50-540132d280ad" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:46:32.528692 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:46:32.528654    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw" podUID="16fc6217-3711-4d08-ad50-540132d280ad" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:46:42.529177 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:46:42.529141    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw" podUID="16fc6217-3711-4d08-ad50-540132d280ad" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:46:48.559391 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:46:48.559338    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:46:49.531915 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:46:49.531882    5609 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-8b97550813bf: secret "agc-cred-8b97550813bf" not found
Nov 04 03:46:49.531915 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:46:49.531904    5609 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-df4cf9975-hk75w: secret "agc-cred-8b97550813bf" not found
Nov 04 03:46:49.532168 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:46:49.531955    5609 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d-provider-credentials podName:0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d nodeName:}" failed. No retries permitted until 2026-11-04 03:48:51.531941094 +0000 UTC m=+1874.108593986 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d-provider-credentials") pod "payload-processing-external-model-df4cf9975-hk75w" (UID: "0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d") : secret "agc-cred-8b97550813bf" not found
Nov 04 03:46:52.528814 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:46:52.528779    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw" podUID="16fc6217-3711-4d08-ad50-540132d280ad" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:47:02.528978 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:47:02.528936    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw" podUID="16fc6217-3711-4d08-ad50-540132d280ad" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:47:12.528758 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:47:12.528718    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw" podUID="16fc6217-3711-4d08-ad50-540132d280ad" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:47:18.558749 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:47:18.558711    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:47:22.528682 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:47:22.528646    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw" podUID="16fc6217-3711-4d08-ad50-540132d280ad" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:47:32.528954 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:47:32.528913    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw" podUID="16fc6217-3711-4d08-ad50-540132d280ad" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:47:35.529210 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:47:35.529171    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw" podUID="16fc6217-3711-4d08-ad50-540132d280ad" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:47:37.907930 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:47:37.907899    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-p9989_068619f8-7fbb-4b22-ac7d-a1f144e084c0/console-operator/2.log"
Nov 04 03:47:37.915455 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:47:37.915432    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vmw5z_21a42913-5c4e-4057-aae5-2804038f16e4/ovn-acl-logging/0.log"
Nov 04 03:47:37.926545 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:47:37.926522    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-p9989_068619f8-7fbb-4b22-ac7d-a1f144e084c0/console-operator/2.log"
Nov 04 03:47:37.932994 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:47:37.932977    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vmw5z_21a42913-5c4e-4057-aae5-2804038f16e4/ovn-acl-logging/0.log"
Nov 04 03:47:45.529897 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:47:45.529860    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw" podUID="16fc6217-3711-4d08-ad50-540132d280ad" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:47:46.048095 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:47:46.048060    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-172a9554-sqjkm"]
Nov 04 03:47:46.094564 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:47:46.094532    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-172a9554-sqjkm"]
Nov 04 03:47:47.533893 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:47:47.533856    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9320f6ca-389b-4857-86b3-5336f90445ce" path="/var/lib/kubelet/pods/9320f6ca-389b-4857-86b3-5336f90445ce/volumes"
Nov 04 03:47:48.558955 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:47:48.558921    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:47:55.529486 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:47:55.529443    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw" podUID="16fc6217-3711-4d08-ad50-540132d280ad" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:47:59.047982 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:47:59.047947    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-785569ce-k8zzt"]
Nov 04 03:47:59.094480 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:47:59.094445    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-785569ce-k8zzt"]
Nov 04 03:47:59.533803 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:47:59.533770    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="30577123-d648-4623-ba41-1443be46c11e" path="/var/lib/kubelet/pods/30577123-d648-4623-ba41-1443be46c11e/volumes"
Nov 04 03:48:05.529273 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:05.529237    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw" podUID="16fc6217-3711-4d08-ad50-540132d280ad" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:48:09.047545 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:09.047509    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-82e703ce-slsch"]
Nov 04 03:48:09.095046 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:09.094994    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-82e703ce-slsch"]
Nov 04 03:48:09.534338 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:09.534303    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="98eb52e2-860b-4b49-9cc9-7b971fdb5224" path="/var/lib/kubelet/pods/98eb52e2-860b-4b49-9cc9-7b971fdb5224/volumes"
Nov 04 03:48:15.529330 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:15.529291    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw" podUID="16fc6217-3711-4d08-ad50-540132d280ad" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:48:18.560099 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:48:18.560061    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:48:25.529228 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:25.529190    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw" podUID="16fc6217-3711-4d08-ad50-540132d280ad" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:48:26.048296 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:26.048265    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2c87d0f1-f6pvr"]
Nov 04 03:48:26.095207 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:26.095176    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2c87d0f1-f6pvr"]
Nov 04 03:48:27.537344 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:27.536942    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fc5519c5-c7a9-40e0-8eff-b4d0b8887f32" path="/var/lib/kubelet/pods/fc5519c5-c7a9-40e0-8eff-b4d0b8887f32/volumes"
Nov 04 03:48:35.529149 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:35.529103    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw" podUID="16fc6217-3711-4d08-ad50-540132d280ad" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:48:36.529243 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:36.529205    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw" podUID="16fc6217-3711-4d08-ad50-540132d280ad" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:48:40.047134 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:40.047028    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9ec8eb24-glglh"]
Nov 04 03:48:40.094327 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:40.094296    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9ec8eb24-glglh"]
Nov 04 03:48:41.534361 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:41.534318    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3" path="/var/lib/kubelet/pods/3a0d0bd8-f6ea-4db3-9c80-6ea67688d0e3/volumes"
Nov 04 03:48:46.529613 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:46.529578    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw" podUID="16fc6217-3711-4d08-ad50-540132d280ad" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:48:48.561172 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:48:48.561134    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:48:51.264152 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:51.264123    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0973bc47-z9rdn"]
Nov 04 03:48:51.264603 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:51.264588    5609 cpu_manager.go:414] "containerMap: removing container" podUID="97ee046b-2ad6-44b4-9e1f-7286d8988b2c" containerName="cleanup"
Nov 04 03:48:51.264650 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:51.264604    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="97ee046b-2ad6-44b4-9e1f-7286d8988b2c" containerName="cleanup"
Nov 04 03:48:51.264650 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:51.264624    5609 cpu_manager.go:414] "containerMap: removing container" podUID="97ee046b-2ad6-44b4-9e1f-7286d8988b2c" containerName="cleanup"
Nov 04 03:48:51.264650 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:51.264629    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="97ee046b-2ad6-44b4-9e1f-7286d8988b2c" containerName="cleanup"
Nov 04 03:48:51.264650 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:51.264639    5609 cpu_manager.go:414] "containerMap: removing container" podUID="0adb4153-ff09-4f78-b28b-d976700f145f" containerName="cleanup"
Nov 04 03:48:51.264650 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:51.264644    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0adb4153-ff09-4f78-b28b-d976700f145f" containerName="cleanup"
Nov 04 03:48:51.264650 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:51.264651    5609 cpu_manager.go:414] "containerMap: removing container" podUID="0adb4153-ff09-4f78-b28b-d976700f145f" containerName="cleanup"
Nov 04 03:48:51.264828 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:51.264656    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0adb4153-ff09-4f78-b28b-d976700f145f" containerName="cleanup"
Nov 04 03:48:51.264828 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:51.264671    5609 cpu_manager.go:414] "containerMap: removing container" podUID="0adb4153-ff09-4f78-b28b-d976700f145f" containerName="cleanup"
Nov 04 03:48:51.264828 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:51.264676    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0adb4153-ff09-4f78-b28b-d976700f145f" containerName="cleanup"
Nov 04 03:48:51.264828 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:51.264750    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="97ee046b-2ad6-44b4-9e1f-7286d8988b2c" containerName="cleanup"
Nov 04 03:48:51.264828 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:51.264764    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="97ee046b-2ad6-44b4-9e1f-7286d8988b2c" containerName="cleanup"
Nov 04 03:48:51.264828 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:51.264784    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="0adb4153-ff09-4f78-b28b-d976700f145f" containerName="cleanup"
Nov 04 03:48:51.264828 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:51.264792    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="0adb4153-ff09-4f78-b28b-d976700f145f" containerName="cleanup"
Nov 04 03:48:51.264828 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:51.264805    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="0adb4153-ff09-4f78-b28b-d976700f145f" containerName="cleanup"
Nov 04 03:48:51.269068 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:51.269049    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0973bc47-z9rdn"
Nov 04 03:48:51.318692 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:51.318639    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 03:48:51.324444 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:51.324392    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0ae0eefa-fbd0-4073-ab76-a291aedd86e1-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0973bc47-z9rdn\" (UID: \"0ae0eefa-fbd0-4073-ab76-a291aedd86e1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0973bc47-z9rdn"
Nov 04 03:48:51.324559 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:51.324519    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0ae0eefa-fbd0-4073-ab76-a291aedd86e1-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0973bc47-z9rdn\" (UID: \"0ae0eefa-fbd0-4073-ab76-a291aedd86e1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0973bc47-z9rdn"
Nov 04 03:48:51.324598 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:51.324574    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzn2v\" (UniqueName: \"kubernetes.io/projected/0ae0eefa-fbd0-4073-ab76-a291aedd86e1-kube-api-access-rzn2v\") pod \"maas-api-revoke-sub-0973bc47-z9rdn\" (UID: \"0ae0eefa-fbd0-4073-ab76-a291aedd86e1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0973bc47-z9rdn"
Nov 04 03:48:51.365893 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:51.365860    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x9cnx\""
Nov 04 03:48:51.368653 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:51.368627    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0973bc47-z9rdn"]
Nov 04 03:48:51.425838 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:51.425808    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0ae0eefa-fbd0-4073-ab76-a291aedd86e1-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0973bc47-z9rdn\" (UID: \"0ae0eefa-fbd0-4073-ab76-a291aedd86e1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0973bc47-z9rdn"
Nov 04 03:48:51.426002 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:51.425867    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzn2v\" (UniqueName: \"kubernetes.io/projected/0ae0eefa-fbd0-4073-ab76-a291aedd86e1-kube-api-access-rzn2v\") pod \"maas-api-revoke-sub-0973bc47-z9rdn\" (UID: \"0ae0eefa-fbd0-4073-ab76-a291aedd86e1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0973bc47-z9rdn"
Nov 04 03:48:51.426002 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:51.425974    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0ae0eefa-fbd0-4073-ab76-a291aedd86e1-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0973bc47-z9rdn\" (UID: \"0ae0eefa-fbd0-4073-ab76-a291aedd86e1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0973bc47-z9rdn"
Nov 04 03:48:51.426755 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:51.426731    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0ae0eefa-fbd0-4073-ab76-a291aedd86e1-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0973bc47-z9rdn\" (UID: \"0ae0eefa-fbd0-4073-ab76-a291aedd86e1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0973bc47-z9rdn"
Nov 04 03:48:51.428134 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:51.428114    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0ae0eefa-fbd0-4073-ab76-a291aedd86e1-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0973bc47-z9rdn\" (UID: \"0ae0eefa-fbd0-4073-ab76-a291aedd86e1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0973bc47-z9rdn"
Nov 04 03:48:51.483867 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:51.483841    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzn2v\" (UniqueName: \"kubernetes.io/projected/0ae0eefa-fbd0-4073-ab76-a291aedd86e1-kube-api-access-rzn2v\") pod \"maas-api-revoke-sub-0973bc47-z9rdn\" (UID: \"0ae0eefa-fbd0-4073-ab76-a291aedd86e1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0973bc47-z9rdn"
Nov 04 03:48:51.582631 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:51.582555    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0973bc47-z9rdn"
Nov 04 03:48:51.628126 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:48:51.628091    5609 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-8b97550813bf: secret "agc-cred-8b97550813bf" not found
Nov 04 03:48:51.628126 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:48:51.628121    5609 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-df4cf9975-hk75w: secret "agc-cred-8b97550813bf" not found
Nov 04 03:48:51.628359 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:48:51.628173    5609 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d-provider-credentials podName:0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d nodeName:}" failed. No retries permitted until 2026-11-04 03:50:53.628156131 +0000 UTC m=+1996.204809027 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d-provider-credentials") pod "payload-processing-external-model-df4cf9975-hk75w" (UID: "0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d") : secret "agc-cred-8b97550813bf" not found
Nov 04 03:48:51.958491 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:51.958459    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0973bc47-z9rdn"]
Nov 04 03:48:51.963590 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:48:51.963561    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ae0eefa_fbd0_4073_ab76_a291aedd86e1.slice/crio-d5f19f7f60e775b08e79b9dfa549d27ef621bd1c5723f3cef73630795b7dff29 WatchSource:0}: Error finding container d5f19f7f60e775b08e79b9dfa549d27ef621bd1c5723f3cef73630795b7dff29: Status 404 returned error can't find the container with id d5f19f7f60e775b08e79b9dfa549d27ef621bd1c5723f3cef73630795b7dff29
Nov 04 03:48:52.567172 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:52.567090    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0973bc47-z9rdn" event={"ID":"0ae0eefa-fbd0-4073-ab76-a291aedd86e1","Type":"ContainerStarted","Data":"9c322d6147d675b4a592326e7b946260e6c3d8af06013438a6ba3bc3bf9d29ff"}
Nov 04 03:48:52.567172 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:52.567144    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0973bc47-z9rdn" event={"ID":"0ae0eefa-fbd0-4073-ab76-a291aedd86e1","Type":"ContainerStarted","Data":"d5f19f7f60e775b08e79b9dfa549d27ef621bd1c5723f3cef73630795b7dff29"}
Nov 04 03:48:52.680638 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:52.680593    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0973bc47-z9rdn" podStartSLOduration=1.6805819180000001 podStartE2EDuration="1.680581918s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:48:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:48:52.680195947 +0000 UTC m=+1875.256848860" watchObservedRunningTime="2026-11-04 03:48:52.680581918 +0000 UTC m=+1875.257234831"
Nov 04 03:48:53.574408 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:53.574375    5609 generic.go:356] "Generic (PLEG): container finished" podID="0ae0eefa-fbd0-4073-ab76-a291aedd86e1" containerID="9c322d6147d675b4a592326e7b946260e6c3d8af06013438a6ba3bc3bf9d29ff" exitCode=0
Nov 04 03:48:53.574825 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:53.574431    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0973bc47-z9rdn" event={"ID":"0ae0eefa-fbd0-4073-ab76-a291aedd86e1","Type":"ContainerDied","Data":"9c322d6147d675b4a592326e7b946260e6c3d8af06013438a6ba3bc3bf9d29ff"}
Nov 04 03:48:54.716573 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:54.716550    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0973bc47-z9rdn"
Nov 04 03:48:54.757246 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:54.757217    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0ae0eefa-fbd0-4073-ab76-a291aedd86e1-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0ae0eefa-fbd0-4073-ab76-a291aedd86e1-maas-api-xks-ca\") pod \"0ae0eefa-fbd0-4073-ab76-a291aedd86e1\" (UID: \"0ae0eefa-fbd0-4073-ab76-a291aedd86e1\") "
Nov 04 03:48:54.757364 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:54.757291    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0ae0eefa-fbd0-4073-ab76-a291aedd86e1-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0ae0eefa-fbd0-4073-ab76-a291aedd86e1-maas-api-openshift-ca\") pod \"0ae0eefa-fbd0-4073-ab76-a291aedd86e1\" (UID: \"0ae0eefa-fbd0-4073-ab76-a291aedd86e1\") "
Nov 04 03:48:54.757364 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:54.757316    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0ae0eefa-fbd0-4073-ab76-a291aedd86e1-kube-api-access-rzn2v\" (UniqueName: \"kubernetes.io/projected/0ae0eefa-fbd0-4073-ab76-a291aedd86e1-kube-api-access-rzn2v\") pod \"0ae0eefa-fbd0-4073-ab76-a291aedd86e1\" (UID: \"0ae0eefa-fbd0-4073-ab76-a291aedd86e1\") "
Nov 04 03:48:54.757717 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:54.757689    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ae0eefa-fbd0-4073-ab76-a291aedd86e1-maas-api-openshift-ca" pod "0ae0eefa-fbd0-4073-ab76-a291aedd86e1" (UID: "0ae0eefa-fbd0-4073-ab76-a291aedd86e1"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:48:54.759314 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:54.759290    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ae0eefa-fbd0-4073-ab76-a291aedd86e1-maas-api-xks-ca" pod "0ae0eefa-fbd0-4073-ab76-a291aedd86e1" (UID: "0ae0eefa-fbd0-4073-ab76-a291aedd86e1"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:48:54.759495 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:54.759472    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ae0eefa-fbd0-4073-ab76-a291aedd86e1-kube-api-access-rzn2v" pod "0ae0eefa-fbd0-4073-ab76-a291aedd86e1" (UID: "0ae0eefa-fbd0-4073-ab76-a291aedd86e1"). InnerVolumeSpecName "kube-api-access-rzn2v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:48:54.858848 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:54.858772    5609 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0ae0eefa-fbd0-4073-ab76-a291aedd86e1-maas-api-openshift-ca\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:48:54.858848 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:54.858809    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rzn2v\" (UniqueName: \"kubernetes.io/projected/0ae0eefa-fbd0-4073-ab76-a291aedd86e1-kube-api-access-rzn2v\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:48:54.858848 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:54.858824    5609 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0ae0eefa-fbd0-4073-ab76-a291aedd86e1-maas-api-xks-ca\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:48:55.589438 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:55.589383    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0973bc47-z9rdn"
Nov 04 03:48:55.589613 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:55.589395    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0973bc47-z9rdn" event={"ID":"0ae0eefa-fbd0-4073-ab76-a291aedd86e1","Type":"ContainerDied","Data":"d5f19f7f60e775b08e79b9dfa549d27ef621bd1c5723f3cef73630795b7dff29"}
Nov 04 03:48:55.589613 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:55.589558    5609 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d5f19f7f60e775b08e79b9dfa549d27ef621bd1c5723f3cef73630795b7dff29"
Nov 04 03:48:56.530144 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:48:56.530101    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw" podUID="16fc6217-3711-4d08-ad50-540132d280ad" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:49:04.869425 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:04.869378    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-df4cf9975-hk75w"]
Nov 04 03:49:04.916275 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:04.916241    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-7489856dc7-h45cw"]
Nov 04 03:49:04.917630 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:04.917595    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw" podUID="16fc6217-3711-4d08-ad50-540132d280ad" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:49:05.045925 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:05.045886    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b62a1bcc-c6kx9"]
Nov 04 03:49:05.048843 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:05.048815    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b62a1bcc-c6kx9"]
Nov 04 03:49:05.533636 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:05.533603    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="00e7f455-7909-4e74-a642-455cfd943ae1" path="/var/lib/kubelet/pods/00e7f455-7909-4e74-a642-455cfd943ae1/volumes"
Nov 04 03:49:09.870386 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:09.870343    5609 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-df4cf9975-hk75w" podUID="0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d" containerName="payload-processing" containerID="cri-o://95b5da072706d8be768c408c6bc1c096a94ce103b5c40952474ef3f0f38e845c" gracePeriod=25
Nov 04 03:49:09.916716 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:09.916679    5609 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw" podUID="16fc6217-3711-4d08-ad50-540132d280ad" containerName="payload-processing" containerID="cri-o://34e7845f6d1bb1e0af87bb84f9f332f46cd5c86dd6b73b20b5209b16027768c6" gracePeriod=25
Nov 04 03:49:10.268642 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:10.268614    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-df4cf9975-hk75w"
Nov 04 03:49:10.275571 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:10.275546    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw"
Nov 04 03:49:10.341883 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:10.341845    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/16fc6217-3711-4d08-ad50-540132d280ad-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/16fc6217-3711-4d08-ad50-540132d280ad-routing-overlay-volume\") pod \"16fc6217-3711-4d08-ad50-540132d280ad\" (UID: \"16fc6217-3711-4d08-ad50-540132d280ad\") "
Nov 04 03:49:10.342086 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:10.341904    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/16fc6217-3711-4d08-ad50-540132d280ad-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16fc6217-3711-4d08-ad50-540132d280ad-plugins-config-volume\") pod \"16fc6217-3711-4d08-ad50-540132d280ad\" (UID: \"16fc6217-3711-4d08-ad50-540132d280ad\") "
Nov 04 03:49:10.342086 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:10.341963    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d-plugins-config-volume\") pod \"0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d\" (UID: \"0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d\") "
Nov 04 03:49:10.342086 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:10.342019    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d-provider-credentials\" (UniqueName: \"kubernetes.io/projected/0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d-provider-credentials\") pod \"0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d\" (UID: \"0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d\") "
Nov 04 03:49:10.342086 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:10.342055    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/16fc6217-3711-4d08-ad50-540132d280ad-provider-credentials\" (UniqueName: \"kubernetes.io/projected/16fc6217-3711-4d08-ad50-540132d280ad-provider-credentials\") pod \"16fc6217-3711-4d08-ad50-540132d280ad\" (UID: \"16fc6217-3711-4d08-ad50-540132d280ad\") "
Nov 04 03:49:10.342327 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:10.342109    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/16fc6217-3711-4d08-ad50-540132d280ad-kube-api-access-n78k9\" (UniqueName: \"kubernetes.io/projected/16fc6217-3711-4d08-ad50-540132d280ad-kube-api-access-n78k9\") pod \"16fc6217-3711-4d08-ad50-540132d280ad\" (UID: \"16fc6217-3711-4d08-ad50-540132d280ad\") "
Nov 04 03:49:10.342327 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:10.342180    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d-routing-overlay-volume\") pod \"0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d\" (UID: \"0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d\") "
Nov 04 03:49:10.342327 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:10.342217    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d-kube-api-access-fsvsr\" (UniqueName: \"kubernetes.io/projected/0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d-kube-api-access-fsvsr\") pod \"0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d\" (UID: \"0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d\") "
Nov 04 03:49:10.343936 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:10.343857    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16fc6217-3711-4d08-ad50-540132d280ad-routing-overlay-volume" pod "16fc6217-3711-4d08-ad50-540132d280ad" (UID: "16fc6217-3711-4d08-ad50-540132d280ad"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:49:10.343936 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:10.343913    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d-plugins-config-volume" pod "0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d" (UID: "0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:49:10.344814 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:10.344761    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d-routing-overlay-volume" pod "0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d" (UID: "0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:49:10.346442 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:10.346307    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16fc6217-3711-4d08-ad50-540132d280ad-plugins-config-volume" pod "16fc6217-3711-4d08-ad50-540132d280ad" (UID: "16fc6217-3711-4d08-ad50-540132d280ad"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:49:10.346938 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:10.346912    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d-kube-api-access-fsvsr" pod "0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d" (UID: "0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d"). InnerVolumeSpecName "kube-api-access-fsvsr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:49:10.347041 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:10.347023    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16fc6217-3711-4d08-ad50-540132d280ad-kube-api-access-n78k9" pod "16fc6217-3711-4d08-ad50-540132d280ad" (UID: "16fc6217-3711-4d08-ad50-540132d280ad"). InnerVolumeSpecName "kube-api-access-n78k9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:49:10.347443 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:10.347384    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16fc6217-3711-4d08-ad50-540132d280ad-provider-credentials" pod "16fc6217-3711-4d08-ad50-540132d280ad" (UID: "16fc6217-3711-4d08-ad50-540132d280ad"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:49:10.348153 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:10.348125    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d-provider-credentials" pod "0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d" (UID: "0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:49:10.443530 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:10.443493    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n78k9\" (UniqueName: \"kubernetes.io/projected/16fc6217-3711-4d08-ad50-540132d280ad-kube-api-access-n78k9\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:49:10.443530 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:10.443530    5609 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d-routing-overlay-volume\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:49:10.443790 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:10.443547    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fsvsr\" (UniqueName: \"kubernetes.io/projected/0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d-kube-api-access-fsvsr\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:49:10.443790 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:10.443564    5609 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/16fc6217-3711-4d08-ad50-540132d280ad-routing-overlay-volume\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:49:10.443790 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:10.443578    5609 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16fc6217-3711-4d08-ad50-540132d280ad-plugins-config-volume\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:49:10.443790 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:10.443593    5609 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d-plugins-config-volume\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:49:10.443790 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:10.443608    5609 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d-provider-credentials\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:49:10.443790 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:10.443622    5609 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/16fc6217-3711-4d08-ad50-540132d280ad-provider-credentials\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:49:10.690424 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:10.689310    5609 generic.go:356] "Generic (PLEG): container finished" podID="0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d" containerID="95b5da072706d8be768c408c6bc1c096a94ce103b5c40952474ef3f0f38e845c" exitCode=0
Nov 04 03:49:10.690424 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:10.689387    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-df4cf9975-hk75w" event={"ID":"0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d","Type":"ContainerDied","Data":"95b5da072706d8be768c408c6bc1c096a94ce103b5c40952474ef3f0f38e845c"}
Nov 04 03:49:10.690424 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:10.689495    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-df4cf9975-hk75w" event={"ID":"0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d","Type":"ContainerDied","Data":"2d8642dae14bec723bbb480618fcfd639d61347c9883476023f37ef6d18feae8"}
Nov 04 03:49:10.690424 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:10.689529    5609 scope.go:122] "RemoveContainer" containerID="95b5da072706d8be768c408c6bc1c096a94ce103b5c40952474ef3f0f38e845c"
Nov 04 03:49:10.690424 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:10.690146    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-df4cf9975-hk75w"
Nov 04 03:49:10.693594 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:10.692371    5609 generic.go:356] "Generic (PLEG): container finished" podID="16fc6217-3711-4d08-ad50-540132d280ad" containerID="34e7845f6d1bb1e0af87bb84f9f332f46cd5c86dd6b73b20b5209b16027768c6" exitCode=0
Nov 04 03:49:10.693594 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:10.692418    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw" event={"ID":"16fc6217-3711-4d08-ad50-540132d280ad","Type":"ContainerDied","Data":"34e7845f6d1bb1e0af87bb84f9f332f46cd5c86dd6b73b20b5209b16027768c6"}
Nov 04 03:49:10.693594 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:10.692457    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw" event={"ID":"16fc6217-3711-4d08-ad50-540132d280ad","Type":"ContainerDied","Data":"fc10dd593fd290159422914687b54c946004e2ac5ea67fa796df7818a164bb71"}
Nov 04 03:49:10.693594 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:10.692524    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-7489856dc7-h45cw"
Nov 04 03:49:10.722546 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:10.722523    5609 scope.go:122] "RemoveContainer" containerID="95b5da072706d8be768c408c6bc1c096a94ce103b5c40952474ef3f0f38e845c"
Nov 04 03:49:10.722959 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:49:10.722927    5609 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95b5da072706d8be768c408c6bc1c096a94ce103b5c40952474ef3f0f38e845c\": container with ID starting with 95b5da072706d8be768c408c6bc1c096a94ce103b5c40952474ef3f0f38e845c not found: ID does not exist" containerID="95b5da072706d8be768c408c6bc1c096a94ce103b5c40952474ef3f0f38e845c"
Nov 04 03:49:10.723073 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:10.722971    5609 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95b5da072706d8be768c408c6bc1c096a94ce103b5c40952474ef3f0f38e845c"} err="failed to get container status \"95b5da072706d8be768c408c6bc1c096a94ce103b5c40952474ef3f0f38e845c\": rpc error: code = NotFound desc = could not find container \"95b5da072706d8be768c408c6bc1c096a94ce103b5c40952474ef3f0f38e845c\": container with ID starting with 95b5da072706d8be768c408c6bc1c096a94ce103b5c40952474ef3f0f38e845c not found: ID does not exist"
Nov 04 03:49:10.723073 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:10.722998    5609 scope.go:122] "RemoveContainer" containerID="34e7845f6d1bb1e0af87bb84f9f332f46cd5c86dd6b73b20b5209b16027768c6"
Nov 04 03:49:10.736678 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:10.736635    5609 scope.go:122] "RemoveContainer" containerID="34e7845f6d1bb1e0af87bb84f9f332f46cd5c86dd6b73b20b5209b16027768c6"
Nov 04 03:49:10.737181 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:49:10.737105    5609 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34e7845f6d1bb1e0af87bb84f9f332f46cd5c86dd6b73b20b5209b16027768c6\": container with ID starting with 34e7845f6d1bb1e0af87bb84f9f332f46cd5c86dd6b73b20b5209b16027768c6 not found: ID does not exist" containerID="34e7845f6d1bb1e0af87bb84f9f332f46cd5c86dd6b73b20b5209b16027768c6"
Nov 04 03:49:10.737181 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:10.737137    5609 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34e7845f6d1bb1e0af87bb84f9f332f46cd5c86dd6b73b20b5209b16027768c6"} err="failed to get container status \"34e7845f6d1bb1e0af87bb84f9f332f46cd5c86dd6b73b20b5209b16027768c6\": rpc error: code = NotFound desc = could not find container \"34e7845f6d1bb1e0af87bb84f9f332f46cd5c86dd6b73b20b5209b16027768c6\": container with ID starting with 34e7845f6d1bb1e0af87bb84f9f332f46cd5c86dd6b73b20b5209b16027768c6 not found: ID does not exist"
Nov 04 03:49:10.851781 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:10.850297    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-df4cf9975-hk75w"]
Nov 04 03:49:10.857045 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:10.857012    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-df4cf9975-hk75w"]
Nov 04 03:49:10.966718 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:10.966638    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-7489856dc7-h45cw"]
Nov 04 03:49:10.970806 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:10.970781    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-7489856dc7-h45cw"]
Nov 04 03:49:11.552674 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:11.551505    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d" path="/var/lib/kubelet/pods/0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d/volumes"
Nov 04 03:49:11.552674 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:11.552075    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="16fc6217-3711-4d08-ad50-540132d280ad" path="/var/lib/kubelet/pods/16fc6217-3711-4d08-ad50-540132d280ad/volumes"
Nov 04 03:49:18.564484 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:49:18.564370    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:49:22.048478 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:22.048445    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b349fc13-fccql"]
Nov 04 03:49:22.053506 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:22.053483    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b349fc13-fccql"]
Nov 04 03:49:23.533835 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:23.533800    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="572347a8-4105-40e9-8804-e77cd582c936" path="/var/lib/kubelet/pods/572347a8-4105-40e9-8804-e77cd582c936/volumes"
Nov 04 03:49:23.766657 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:23.766620    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-6d170d-5847588f6c-sp8v2"]
Nov 04 03:49:28.767656 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:28.767602    5609 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6d170d-5847588f6c-sp8v2" podUID="4da4148c-c27f-4b6a-af35-543b1f81fb15" containerName="payload-pre-processing" containerID="cri-o://2abe406ac007c72bf9f702b1cf0a42734630f1509d2514943c0c587e3d5eb1e7" gracePeriod=25
Nov 04 03:49:29.017151 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:29.017126    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6d170d-5847588f6c-sp8v2"
Nov 04 03:49:29.162908 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:29.162828    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4da4148c-c27f-4b6a-af35-543b1f81fb15-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4da4148c-c27f-4b6a-af35-543b1f81fb15-plugins-config-volume\") pod \"4da4148c-c27f-4b6a-af35-543b1f81fb15\" (UID: \"4da4148c-c27f-4b6a-af35-543b1f81fb15\") "
Nov 04 03:49:29.162908 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:29.162882    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4da4148c-c27f-4b6a-af35-543b1f81fb15-kube-api-access-52ckd\" (UniqueName: \"kubernetes.io/projected/4da4148c-c27f-4b6a-af35-543b1f81fb15-kube-api-access-52ckd\") pod \"4da4148c-c27f-4b6a-af35-543b1f81fb15\" (UID: \"4da4148c-c27f-4b6a-af35-543b1f81fb15\") "
Nov 04 03:49:29.163327 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:29.163298    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4da4148c-c27f-4b6a-af35-543b1f81fb15-plugins-config-volume" pod "4da4148c-c27f-4b6a-af35-543b1f81fb15" (UID: "4da4148c-c27f-4b6a-af35-543b1f81fb15"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:49:29.164960 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:29.164935    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4da4148c-c27f-4b6a-af35-543b1f81fb15-kube-api-access-52ckd" pod "4da4148c-c27f-4b6a-af35-543b1f81fb15" (UID: "4da4148c-c27f-4b6a-af35-543b1f81fb15"). InnerVolumeSpecName "kube-api-access-52ckd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:49:29.264429 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:29.264367    5609 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4da4148c-c27f-4b6a-af35-543b1f81fb15-plugins-config-volume\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:49:29.264429 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:29.264424    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-52ckd\" (UniqueName: \"kubernetes.io/projected/4da4148c-c27f-4b6a-af35-543b1f81fb15-kube-api-access-52ckd\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:49:29.875118 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:29.875034    5609 generic.go:356] "Generic (PLEG): container finished" podID="4da4148c-c27f-4b6a-af35-543b1f81fb15" containerID="2abe406ac007c72bf9f702b1cf0a42734630f1509d2514943c0c587e3d5eb1e7" exitCode=0
Nov 04 03:49:29.875118 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:29.875082    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6d170d-5847588f6c-sp8v2"
Nov 04 03:49:29.875673 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:29.875119    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6d170d-5847588f6c-sp8v2" event={"ID":"4da4148c-c27f-4b6a-af35-543b1f81fb15","Type":"ContainerDied","Data":"2abe406ac007c72bf9f702b1cf0a42734630f1509d2514943c0c587e3d5eb1e7"}
Nov 04 03:49:29.875673 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:29.875170    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6d170d-5847588f6c-sp8v2" event={"ID":"4da4148c-c27f-4b6a-af35-543b1f81fb15","Type":"ContainerDied","Data":"51e2f43d0a9d01752cb299e2453e1963dd99ee3326349a08a31cba5b61550fc1"}
Nov 04 03:49:29.875673 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:29.875182    5609 scope.go:122] "RemoveContainer" containerID="2abe406ac007c72bf9f702b1cf0a42734630f1509d2514943c0c587e3d5eb1e7"
Nov 04 03:49:29.918790 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:29.918761    5609 scope.go:122] "RemoveContainer" containerID="2abe406ac007c72bf9f702b1cf0a42734630f1509d2514943c0c587e3d5eb1e7"
Nov 04 03:49:29.919199 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:49:29.919175    5609 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2abe406ac007c72bf9f702b1cf0a42734630f1509d2514943c0c587e3d5eb1e7\": container with ID starting with 2abe406ac007c72bf9f702b1cf0a42734630f1509d2514943c0c587e3d5eb1e7 not found: ID does not exist" containerID="2abe406ac007c72bf9f702b1cf0a42734630f1509d2514943c0c587e3d5eb1e7"
Nov 04 03:49:29.919290 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:29.919213    5609 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2abe406ac007c72bf9f702b1cf0a42734630f1509d2514943c0c587e3d5eb1e7"} err="failed to get container status \"2abe406ac007c72bf9f702b1cf0a42734630f1509d2514943c0c587e3d5eb1e7\": rpc error: code = NotFound desc = could not find container \"2abe406ac007c72bf9f702b1cf0a42734630f1509d2514943c0c587e3d5eb1e7\": container with ID starting with 2abe406ac007c72bf9f702b1cf0a42734630f1509d2514943c0c587e3d5eb1e7 not found: ID does not exist"
Nov 04 03:49:30.031875 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:30.031839    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-6d170d-5847588f6c-sp8v2"]
Nov 04 03:49:30.036614 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:30.036585    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-6d170d-5847588f6c-sp8v2"]
Nov 04 03:49:31.534266 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:31.534227    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4da4148c-c27f-4b6a-af35-543b1f81fb15" path="/var/lib/kubelet/pods/4da4148c-c27f-4b6a-af35-543b1f81fb15/volumes"
Nov 04 03:49:37.905752 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:37.905719    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9"]
Nov 04 03:49:37.906132 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:37.905919    5609 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9" podUID="c8d54648-2395-478b-940e-fca3a4deb1b1" containerName="istio-proxy" containerID="cri-o://fdb46f7c87147cecb74f89c8179759932224f4ca5e15b25c80b91781c74912ad" gracePeriod=30
Nov 04 03:49:38.149149 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:38.149112    5609 scope.go:122] "RemoveContainer" containerID="5f63a717e08e6bcb32b14bee625d7388c48b47fe6d2edb88af0dfe9b0e465db6"
Nov 04 03:49:38.162099 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:38.162072    5609 scope.go:122] "RemoveContainer" containerID="eb36c275241083a786c84d8d16a3d22ee7fb8d6ef6df36e4a22a0e4ceca1e888"
Nov 04 03:49:38.170799 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:38.170776    5609 scope.go:122] "RemoveContainer" containerID="0b913e2606895e69b7677ea3bc704589fd8e815b23a9c2e5e909e740d4cb80d2"
Nov 04 03:49:38.179184 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:38.179164    5609 scope.go:122] "RemoveContainer" containerID="dd5e3127ca8be7a4618218a2b8256e80ca2fadd3ff92c9e39c4712cdc2024e3b"
Nov 04 03:49:38.188060 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:38.188038    5609 scope.go:122] "RemoveContainer" containerID="ec9b9b1ace9c5bcc1dee5f35924095e9504f310750500eb37bc79b3da6612991"
Nov 04 03:49:43.160206 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:43.160175    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9"
Nov 04 03:49:43.231722 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:43.231694    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c8d54648-2395-478b-940e-fca3a4deb1b1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8d54648-2395-478b-940e-fca3a4deb1b1-workload-certs\") pod \"c8d54648-2395-478b-940e-fca3a4deb1b1\" (UID: \"c8d54648-2395-478b-940e-fca3a4deb1b1\") "
Nov 04 03:49:43.231881 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:43.231751    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c8d54648-2395-478b-940e-fca3a4deb1b1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8d54648-2395-478b-940e-fca3a4deb1b1-workload-socket\") pod \"c8d54648-2395-478b-940e-fca3a4deb1b1\" (UID: \"c8d54648-2395-478b-940e-fca3a4deb1b1\") "
Nov 04 03:49:43.231881 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:43.231780    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c8d54648-2395-478b-940e-fca3a4deb1b1-istio-token\" (UniqueName: \"kubernetes.io/projected/c8d54648-2395-478b-940e-fca3a4deb1b1-istio-token\") pod \"c8d54648-2395-478b-940e-fca3a4deb1b1\" (UID: \"c8d54648-2395-478b-940e-fca3a4deb1b1\") "
Nov 04 03:49:43.231881 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:43.231818    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c8d54648-2395-478b-940e-fca3a4deb1b1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8d54648-2395-478b-940e-fca3a4deb1b1-istio-podinfo\") pod \"c8d54648-2395-478b-940e-fca3a4deb1b1\" (UID: \"c8d54648-2395-478b-940e-fca3a4deb1b1\") "
Nov 04 03:49:43.231881 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:43.231852    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c8d54648-2395-478b-940e-fca3a4deb1b1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8d54648-2395-478b-940e-fca3a4deb1b1-istiod-ca-cert\") pod \"c8d54648-2395-478b-940e-fca3a4deb1b1\" (UID: \"c8d54648-2395-478b-940e-fca3a4deb1b1\") "
Nov 04 03:49:43.232092 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:43.231910    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c8d54648-2395-478b-940e-fca3a4deb1b1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8d54648-2395-478b-940e-fca3a4deb1b1-credential-socket\") pod \"c8d54648-2395-478b-940e-fca3a4deb1b1\" (UID: \"c8d54648-2395-478b-940e-fca3a4deb1b1\") "
Nov 04 03:49:43.232092 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:43.231944    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c8d54648-2395-478b-940e-fca3a4deb1b1-kube-api-access-z6zvl\" (UniqueName: \"kubernetes.io/projected/c8d54648-2395-478b-940e-fca3a4deb1b1-kube-api-access-z6zvl\") pod \"c8d54648-2395-478b-940e-fca3a4deb1b1\" (UID: \"c8d54648-2395-478b-940e-fca3a4deb1b1\") "
Nov 04 03:49:43.232092 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:43.231961    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8d54648-2395-478b-940e-fca3a4deb1b1-workload-certs" pod "c8d54648-2395-478b-940e-fca3a4deb1b1" (UID: "c8d54648-2395-478b-940e-fca3a4deb1b1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:49:43.232092 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:43.232048    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c8d54648-2395-478b-940e-fca3a4deb1b1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8d54648-2395-478b-940e-fca3a4deb1b1-istio-data\") pod \"c8d54648-2395-478b-940e-fca3a4deb1b1\" (UID: \"c8d54648-2395-478b-940e-fca3a4deb1b1\") "
Nov 04 03:49:43.232092 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:43.232086    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c8d54648-2395-478b-940e-fca3a4deb1b1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8d54648-2395-478b-940e-fca3a4deb1b1-istio-envoy\") pod \"c8d54648-2395-478b-940e-fca3a4deb1b1\" (UID: \"c8d54648-2395-478b-940e-fca3a4deb1b1\") "
Nov 04 03:49:43.232330 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:43.232083    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8d54648-2395-478b-940e-fca3a4deb1b1-workload-socket" pod "c8d54648-2395-478b-940e-fca3a4deb1b1" (UID: "c8d54648-2395-478b-940e-fca3a4deb1b1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:49:43.232330 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:43.232228    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8d54648-2395-478b-940e-fca3a4deb1b1-credential-socket" pod "c8d54648-2395-478b-940e-fca3a4deb1b1" (UID: "c8d54648-2395-478b-940e-fca3a4deb1b1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:49:43.232330 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:43.232306    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8d54648-2395-478b-940e-fca3a4deb1b1-istio-data" pod "c8d54648-2395-478b-940e-fca3a4deb1b1" (UID: "c8d54648-2395-478b-940e-fca3a4deb1b1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:49:43.232537 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:43.232330    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8d54648-2395-478b-940e-fca3a4deb1b1-istiod-ca-cert" pod "c8d54648-2395-478b-940e-fca3a4deb1b1" (UID: "c8d54648-2395-478b-940e-fca3a4deb1b1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:49:43.232593 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:43.232548    5609 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8d54648-2395-478b-940e-fca3a4deb1b1-istiod-ca-cert\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:49:43.232593 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:43.232565    5609 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8d54648-2395-478b-940e-fca3a4deb1b1-credential-socket\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:49:43.232593 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:43.232576    5609 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8d54648-2395-478b-940e-fca3a4deb1b1-istio-data\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:49:43.232593 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:43.232588    5609 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8d54648-2395-478b-940e-fca3a4deb1b1-workload-certs\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:49:43.232802 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:43.232600    5609 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8d54648-2395-478b-940e-fca3a4deb1b1-workload-socket\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:49:43.233997 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:43.233975    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c8d54648-2395-478b-940e-fca3a4deb1b1-istio-podinfo" pod "c8d54648-2395-478b-940e-fca3a4deb1b1" (UID: "c8d54648-2395-478b-940e-fca3a4deb1b1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:49:43.234204 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:43.234183    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8d54648-2395-478b-940e-fca3a4deb1b1-istio-envoy" pod "c8d54648-2395-478b-940e-fca3a4deb1b1" (UID: "c8d54648-2395-478b-940e-fca3a4deb1b1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:49:43.234521 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:43.234502    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8d54648-2395-478b-940e-fca3a4deb1b1-istio-token" pod "c8d54648-2395-478b-940e-fca3a4deb1b1" (UID: "c8d54648-2395-478b-940e-fca3a4deb1b1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:49:43.234574 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:43.234508    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8d54648-2395-478b-940e-fca3a4deb1b1-kube-api-access-z6zvl" pod "c8d54648-2395-478b-940e-fca3a4deb1b1" (UID: "c8d54648-2395-478b-940e-fca3a4deb1b1"). InnerVolumeSpecName "kube-api-access-z6zvl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:49:43.333574 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:43.333543    5609 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8d54648-2395-478b-940e-fca3a4deb1b1-istio-token\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:49:43.333574 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:43.333571    5609 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8d54648-2395-478b-940e-fca3a4deb1b1-istio-podinfo\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:49:43.333754 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:43.333586    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z6zvl\" (UniqueName: \"kubernetes.io/projected/c8d54648-2395-478b-940e-fca3a4deb1b1-kube-api-access-z6zvl\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:49:43.333754 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:43.333595    5609 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8d54648-2395-478b-940e-fca3a4deb1b1-istio-envoy\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:49:43.963443 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:43.963385    5609 generic.go:356] "Generic (PLEG): container finished" podID="c8d54648-2395-478b-940e-fca3a4deb1b1" containerID="fdb46f7c87147cecb74f89c8179759932224f4ca5e15b25c80b91781c74912ad" exitCode=0
Nov 04 03:49:43.963621 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:43.963456    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9"
Nov 04 03:49:43.963621 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:43.963472    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9" event={"ID":"c8d54648-2395-478b-940e-fca3a4deb1b1","Type":"ContainerDied","Data":"fdb46f7c87147cecb74f89c8179759932224f4ca5e15b25c80b91781c74912ad"}
Nov 04 03:49:43.963621 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:43.963532    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9" event={"ID":"c8d54648-2395-478b-940e-fca3a4deb1b1","Type":"ContainerDied","Data":"8b0b0822955d50986abbd1c7e643032d4d3477fa130ca2351c9b14f3dab36c60"}
Nov 04 03:49:43.963621 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:43.963555    5609 scope.go:122] "RemoveContainer" containerID="fdb46f7c87147cecb74f89c8179759932224f4ca5e15b25c80b91781c74912ad"
Nov 04 03:49:43.976670 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:43.976332    5609 scope.go:122] "RemoveContainer" containerID="e7b077dbbf2a6d13cee704ad6f4079b50916b3e8cde68b168113efb40b6750f8"
Nov 04 03:49:43.985346 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:43.985330    5609 scope.go:122] "RemoveContainer" containerID="fdb46f7c87147cecb74f89c8179759932224f4ca5e15b25c80b91781c74912ad"
Nov 04 03:49:43.985634 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:49:43.985615    5609 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fdb46f7c87147cecb74f89c8179759932224f4ca5e15b25c80b91781c74912ad\": container with ID starting with fdb46f7c87147cecb74f89c8179759932224f4ca5e15b25c80b91781c74912ad not found: ID does not exist" containerID="fdb46f7c87147cecb74f89c8179759932224f4ca5e15b25c80b91781c74912ad"
Nov 04 03:49:43.985690 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:43.985643    5609 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdb46f7c87147cecb74f89c8179759932224f4ca5e15b25c80b91781c74912ad"} err="failed to get container status \"fdb46f7c87147cecb74f89c8179759932224f4ca5e15b25c80b91781c74912ad\": rpc error: code = NotFound desc = could not find container \"fdb46f7c87147cecb74f89c8179759932224f4ca5e15b25c80b91781c74912ad\": container with ID starting with fdb46f7c87147cecb74f89c8179759932224f4ca5e15b25c80b91781c74912ad not found: ID does not exist"
Nov 04 03:49:43.985690 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:43.985661    5609 scope.go:122] "RemoveContainer" containerID="e7b077dbbf2a6d13cee704ad6f4079b50916b3e8cde68b168113efb40b6750f8"
Nov 04 03:49:43.985901 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:49:43.985886    5609 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7b077dbbf2a6d13cee704ad6f4079b50916b3e8cde68b168113efb40b6750f8\": container with ID starting with e7b077dbbf2a6d13cee704ad6f4079b50916b3e8cde68b168113efb40b6750f8 not found: ID does not exist" containerID="e7b077dbbf2a6d13cee704ad6f4079b50916b3e8cde68b168113efb40b6750f8"
Nov 04 03:49:43.985947 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:43.985903    5609 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7b077dbbf2a6d13cee704ad6f4079b50916b3e8cde68b168113efb40b6750f8"} err="failed to get container status \"e7b077dbbf2a6d13cee704ad6f4079b50916b3e8cde68b168113efb40b6750f8\": rpc error: code = NotFound desc = could not find container \"e7b077dbbf2a6d13cee704ad6f4079b50916b3e8cde68b168113efb40b6750f8\": container with ID starting with e7b077dbbf2a6d13cee704ad6f4079b50916b3e8cde68b168113efb40b6750f8 not found: ID does not exist"
Nov 04 03:49:44.213128 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:44.213092    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9"]
Nov 04 03:49:44.217969 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:44.217906    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-6d170d-openshift-default-86667cff57-m8zh9"]
Nov 04 03:49:45.534814 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:45.534779    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c8d54648-2395-478b-940e-fca3a4deb1b1" path="/var/lib/kubelet/pods/c8d54648-2395-478b-940e-fca3a4deb1b1/volumes"
Nov 04 03:49:48.576751 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:49:48.576706    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:49:54.041931 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:54.041894    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f020be60-4zcsf"]
Nov 04 03:49:54.042579 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:54.042555    5609 cpu_manager.go:414] "containerMap: removing container" podUID="0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d" containerName="payload-processing"
Nov 04 03:49:54.042653 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:54.042582    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d" containerName="payload-processing"
Nov 04 03:49:54.042653 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:54.042608    5609 cpu_manager.go:414] "containerMap: removing container" podUID="4da4148c-c27f-4b6a-af35-543b1f81fb15" containerName="payload-pre-processing"
Nov 04 03:49:54.042653 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:54.042617    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4da4148c-c27f-4b6a-af35-543b1f81fb15" containerName="payload-pre-processing"
Nov 04 03:49:54.042653 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:54.042636    5609 cpu_manager.go:414] "containerMap: removing container" podUID="c8d54648-2395-478b-940e-fca3a4deb1b1" containerName="istio-proxy"
Nov 04 03:49:54.042653 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:54.042645    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c8d54648-2395-478b-940e-fca3a4deb1b1" containerName="istio-proxy"
Nov 04 03:49:54.042816 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:54.042659    5609 cpu_manager.go:414] "containerMap: removing container" podUID="16fc6217-3711-4d08-ad50-540132d280ad" containerName="payload-processing"
Nov 04 03:49:54.042816 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:54.042667    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="16fc6217-3711-4d08-ad50-540132d280ad" containerName="payload-processing"
Nov 04 03:49:54.042816 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:54.042707    5609 cpu_manager.go:414] "containerMap: removing container" podUID="0ae0eefa-fbd0-4073-ab76-a291aedd86e1" containerName="revoke-keys"
Nov 04 03:49:54.042816 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:54.042717    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0ae0eefa-fbd0-4073-ab76-a291aedd86e1" containerName="revoke-keys"
Nov 04 03:49:54.042816 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:54.042737    5609 cpu_manager.go:414] "containerMap: removing container" podUID="97ee046b-2ad6-44b4-9e1f-7286d8988b2c" containerName="cleanup"
Nov 04 03:49:54.042816 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:54.042745    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="97ee046b-2ad6-44b4-9e1f-7286d8988b2c" containerName="cleanup"
Nov 04 03:49:54.042816 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:54.042759    5609 cpu_manager.go:414] "containerMap: removing container" podUID="c8d54648-2395-478b-940e-fca3a4deb1b1" containerName="istio-proxy"
Nov 04 03:49:54.042816 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:54.042767    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c8d54648-2395-478b-940e-fca3a4deb1b1" containerName="istio-proxy"
Nov 04 03:49:54.043040 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:54.042873    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="0c6b6ce7-cf16-44cb-b926-33d44b3e4a1d" containerName="payload-processing"
Nov 04 03:49:54.043040 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:54.042884    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="97ee046b-2ad6-44b4-9e1f-7286d8988b2c" containerName="cleanup"
Nov 04 03:49:54.043040 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:54.042900    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="4da4148c-c27f-4b6a-af35-543b1f81fb15" containerName="payload-pre-processing"
Nov 04 03:49:54.043040 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:54.042918    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="c8d54648-2395-478b-940e-fca3a4deb1b1" containerName="istio-proxy"
Nov 04 03:49:54.043040 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:54.042927    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="16fc6217-3711-4d08-ad50-540132d280ad" containerName="payload-processing"
Nov 04 03:49:54.043040 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:54.042938    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="0ae0eefa-fbd0-4073-ab76-a291aedd86e1" containerName="revoke-keys"
Nov 04 03:49:54.048238 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:54.048217    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f020be60-4zcsf"
Nov 04 03:49:54.097773 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:54.097733    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 03:49:54.144951 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:54.144918    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x9cnx\""
Nov 04 03:49:54.145611 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:54.145588    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/47926061-d052-49d9-bc64-73b38cd951d5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f020be60-4zcsf\" (UID: \"47926061-d052-49d9-bc64-73b38cd951d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f020be60-4zcsf"
Nov 04 03:49:54.145698 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:54.145632    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4qgs\" (UniqueName: \"kubernetes.io/projected/47926061-d052-49d9-bc64-73b38cd951d5-kube-api-access-s4qgs\") pod \"maas-api-revoke-sub-f020be60-4zcsf\" (UID: \"47926061-d052-49d9-bc64-73b38cd951d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f020be60-4zcsf"
Nov 04 03:49:54.145814 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:54.145793    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/47926061-d052-49d9-bc64-73b38cd951d5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f020be60-4zcsf\" (UID: \"47926061-d052-49d9-bc64-73b38cd951d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f020be60-4zcsf"
Nov 04 03:49:54.147748 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:54.147728    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f020be60-4zcsf"]
Nov 04 03:49:54.247442 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:54.247370    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/47926061-d052-49d9-bc64-73b38cd951d5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f020be60-4zcsf\" (UID: \"47926061-d052-49d9-bc64-73b38cd951d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f020be60-4zcsf"
Nov 04 03:49:54.247621 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:54.247457    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4qgs\" (UniqueName: \"kubernetes.io/projected/47926061-d052-49d9-bc64-73b38cd951d5-kube-api-access-s4qgs\") pod \"maas-api-revoke-sub-f020be60-4zcsf\" (UID: \"47926061-d052-49d9-bc64-73b38cd951d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f020be60-4zcsf"
Nov 04 03:49:54.247621 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:54.247533    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/47926061-d052-49d9-bc64-73b38cd951d5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f020be60-4zcsf\" (UID: \"47926061-d052-49d9-bc64-73b38cd951d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f020be60-4zcsf"
Nov 04 03:49:54.248237 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:54.248210    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/47926061-d052-49d9-bc64-73b38cd951d5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f020be60-4zcsf\" (UID: \"47926061-d052-49d9-bc64-73b38cd951d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f020be60-4zcsf"
Nov 04 03:49:54.249933 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:54.249908    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/47926061-d052-49d9-bc64-73b38cd951d5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f020be60-4zcsf\" (UID: \"47926061-d052-49d9-bc64-73b38cd951d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f020be60-4zcsf"
Nov 04 03:49:54.305823 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:54.305762    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4qgs\" (UniqueName: \"kubernetes.io/projected/47926061-d052-49d9-bc64-73b38cd951d5-kube-api-access-s4qgs\") pod \"maas-api-revoke-sub-f020be60-4zcsf\" (UID: \"47926061-d052-49d9-bc64-73b38cd951d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f020be60-4zcsf"
Nov 04 03:49:54.361818 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:54.361780    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f020be60-4zcsf"
Nov 04 03:49:54.733043 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:54.733009    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f020be60-4zcsf"]
Nov 04 03:49:54.738016 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:49:54.737989    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47926061_d052_49d9_bc64_73b38cd951d5.slice/crio-f1bbcdec5681a2dc59a733a5c9d5202ed9ba2f922a68de497de4f0c09e5a8b24 WatchSource:0}: Error finding container f1bbcdec5681a2dc59a733a5c9d5202ed9ba2f922a68de497de4f0c09e5a8b24: Status 404 returned error can't find the container with id f1bbcdec5681a2dc59a733a5c9d5202ed9ba2f922a68de497de4f0c09e5a8b24
Nov 04 03:49:55.130754 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:55.130722    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f020be60-4zcsf" event={"ID":"47926061-d052-49d9-bc64-73b38cd951d5","Type":"ContainerStarted","Data":"35d0cf8e4604044d085f64ea01f8e9a4c19a8a0be3edc9b15ec4031aa8199e54"}
Nov 04 03:49:55.131151 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:55.130772    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f020be60-4zcsf" event={"ID":"47926061-d052-49d9-bc64-73b38cd951d5","Type":"ContainerStarted","Data":"f1bbcdec5681a2dc59a733a5c9d5202ed9ba2f922a68de497de4f0c09e5a8b24"}
Nov 04 03:49:55.242702 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:55.242646    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f020be60-4zcsf" podStartSLOduration=2.24263091 podStartE2EDuration="2.24263091s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:49:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:49:55.240054443 +0000 UTC m=+1937.816707353" watchObservedRunningTime="2026-11-04 03:49:55.24263091 +0000 UTC m=+1937.819283821"
Nov 04 03:49:56.140344 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:56.140304    5609 generic.go:356] "Generic (PLEG): container finished" podID="47926061-d052-49d9-bc64-73b38cd951d5" containerID="35d0cf8e4604044d085f64ea01f8e9a4c19a8a0be3edc9b15ec4031aa8199e54" exitCode=0
Nov 04 03:49:56.140761 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:56.140388    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f020be60-4zcsf" event={"ID":"47926061-d052-49d9-bc64-73b38cd951d5","Type":"ContainerDied","Data":"35d0cf8e4604044d085f64ea01f8e9a4c19a8a0be3edc9b15ec4031aa8199e54"}
Nov 04 03:49:57.273794 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:57.273768    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f020be60-4zcsf"
Nov 04 03:49:57.376279 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:57.376241    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/47926061-d052-49d9-bc64-73b38cd951d5-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/47926061-d052-49d9-bc64-73b38cd951d5-maas-api-openshift-ca\") pod \"47926061-d052-49d9-bc64-73b38cd951d5\" (UID: \"47926061-d052-49d9-bc64-73b38cd951d5\") "
Nov 04 03:49:57.376464 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:57.376298    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/47926061-d052-49d9-bc64-73b38cd951d5-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/47926061-d052-49d9-bc64-73b38cd951d5-maas-api-xks-ca\") pod \"47926061-d052-49d9-bc64-73b38cd951d5\" (UID: \"47926061-d052-49d9-bc64-73b38cd951d5\") "
Nov 04 03:49:57.376547 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:57.376495    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/47926061-d052-49d9-bc64-73b38cd951d5-kube-api-access-s4qgs\" (UniqueName: \"kubernetes.io/projected/47926061-d052-49d9-bc64-73b38cd951d5-kube-api-access-s4qgs\") pod \"47926061-d052-49d9-bc64-73b38cd951d5\" (UID: \"47926061-d052-49d9-bc64-73b38cd951d5\") "
Nov 04 03:49:57.376726 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:57.376693    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47926061-d052-49d9-bc64-73b38cd951d5-maas-api-openshift-ca" pod "47926061-d052-49d9-bc64-73b38cd951d5" (UID: "47926061-d052-49d9-bc64-73b38cd951d5"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:49:57.378728 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:57.378699    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47926061-d052-49d9-bc64-73b38cd951d5-kube-api-access-s4qgs" pod "47926061-d052-49d9-bc64-73b38cd951d5" (UID: "47926061-d052-49d9-bc64-73b38cd951d5"). InnerVolumeSpecName "kube-api-access-s4qgs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:49:57.378826 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:57.378701    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47926061-d052-49d9-bc64-73b38cd951d5-maas-api-xks-ca" pod "47926061-d052-49d9-bc64-73b38cd951d5" (UID: "47926061-d052-49d9-bc64-73b38cd951d5"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:49:57.477950 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:57.477904    5609 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/47926061-d052-49d9-bc64-73b38cd951d5-maas-api-openshift-ca\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:49:57.477950 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:57.477950    5609 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/47926061-d052-49d9-bc64-73b38cd951d5-maas-api-xks-ca\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:49:57.478156 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:57.477964    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s4qgs\" (UniqueName: \"kubernetes.io/projected/47926061-d052-49d9-bc64-73b38cd951d5-kube-api-access-s4qgs\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:49:58.154891 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:58.154863    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f020be60-4zcsf"
Nov 04 03:49:58.155083 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:58.154897    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f020be60-4zcsf" event={"ID":"47926061-d052-49d9-bc64-73b38cd951d5","Type":"ContainerDied","Data":"f1bbcdec5681a2dc59a733a5c9d5202ed9ba2f922a68de497de4f0c09e5a8b24"}
Nov 04 03:49:58.155083 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:49:58.154951    5609 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f1bbcdec5681a2dc59a733a5c9d5202ed9ba2f922a68de497de4f0c09e5a8b24"
Nov 04 03:50:06.349096 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:06.349066    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-89f7957c-gll94"]
Nov 04 03:50:06.349503 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:06.349489    5609 cpu_manager.go:414] "containerMap: removing container" podUID="47926061-d052-49d9-bc64-73b38cd951d5" containerName="revoke-keys"
Nov 04 03:50:06.349550 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:06.349505    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="47926061-d052-49d9-bc64-73b38cd951d5" containerName="revoke-keys"
Nov 04 03:50:06.349597 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:06.349588    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="47926061-d052-49d9-bc64-73b38cd951d5" containerName="revoke-keys"
Nov 04 03:50:06.349631 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:06.349604    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="c8d54648-2395-478b-940e-fca3a4deb1b1" containerName="istio-proxy"
Nov 04 03:50:06.352793 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:06.352767    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-89f7957c-gll94"
Nov 04 03:50:06.402335 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:06.402300    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 03:50:06.449515 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:06.449480    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x9cnx\""
Nov 04 03:50:06.453771 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:06.453747    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-89f7957c-gll94"]
Nov 04 03:50:06.478293 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:06.478260    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bm948\" (UniqueName: \"kubernetes.io/projected/3ee001ad-dfbc-4571-ae40-9058341d8674-kube-api-access-bm948\") pod \"maas-api-revoke-sub-89f7957c-gll94\" (UID: \"3ee001ad-dfbc-4571-ae40-9058341d8674\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-89f7957c-gll94"
Nov 04 03:50:06.478471 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:06.478315    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3ee001ad-dfbc-4571-ae40-9058341d8674-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-89f7957c-gll94\" (UID: \"3ee001ad-dfbc-4571-ae40-9058341d8674\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-89f7957c-gll94"
Nov 04 03:50:06.478471 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:06.478438    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3ee001ad-dfbc-4571-ae40-9058341d8674-maas-api-xks-ca\") pod \"maas-api-revoke-sub-89f7957c-gll94\" (UID: \"3ee001ad-dfbc-4571-ae40-9058341d8674\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-89f7957c-gll94"
Nov 04 03:50:06.579822 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:06.579786    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bm948\" (UniqueName: \"kubernetes.io/projected/3ee001ad-dfbc-4571-ae40-9058341d8674-kube-api-access-bm948\") pod \"maas-api-revoke-sub-89f7957c-gll94\" (UID: \"3ee001ad-dfbc-4571-ae40-9058341d8674\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-89f7957c-gll94"
Nov 04 03:50:06.579991 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:06.579827    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3ee001ad-dfbc-4571-ae40-9058341d8674-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-89f7957c-gll94\" (UID: \"3ee001ad-dfbc-4571-ae40-9058341d8674\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-89f7957c-gll94"
Nov 04 03:50:06.579991 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:06.579867    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3ee001ad-dfbc-4571-ae40-9058341d8674-maas-api-xks-ca\") pod \"maas-api-revoke-sub-89f7957c-gll94\" (UID: \"3ee001ad-dfbc-4571-ae40-9058341d8674\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-89f7957c-gll94"
Nov 04 03:50:06.580597 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:06.580570    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3ee001ad-dfbc-4571-ae40-9058341d8674-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-89f7957c-gll94\" (UID: \"3ee001ad-dfbc-4571-ae40-9058341d8674\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-89f7957c-gll94"
Nov 04 03:50:06.582178 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:06.582158    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3ee001ad-dfbc-4571-ae40-9058341d8674-maas-api-xks-ca\") pod \"maas-api-revoke-sub-89f7957c-gll94\" (UID: \"3ee001ad-dfbc-4571-ae40-9058341d8674\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-89f7957c-gll94"
Nov 04 03:50:06.639971 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:06.639911    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bm948\" (UniqueName: \"kubernetes.io/projected/3ee001ad-dfbc-4571-ae40-9058341d8674-kube-api-access-bm948\") pod \"maas-api-revoke-sub-89f7957c-gll94\" (UID: \"3ee001ad-dfbc-4571-ae40-9058341d8674\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-89f7957c-gll94"
Nov 04 03:50:06.665593 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:06.665565    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-89f7957c-gll94"
Nov 04 03:50:07.045131 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:07.045093    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-89f7957c-gll94"]
Nov 04 03:50:07.049489 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:50:07.049455    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ee001ad_dfbc_4571_ae40_9058341d8674.slice/crio-b0885bc60c08918eb82829ac2305ee7f7ead1629ca8da764c45f5a20624e2a00 WatchSource:0}: Error finding container b0885bc60c08918eb82829ac2305ee7f7ead1629ca8da764c45f5a20624e2a00: Status 404 returned error can't find the container with id b0885bc60c08918eb82829ac2305ee7f7ead1629ca8da764c45f5a20624e2a00
Nov 04 03:50:07.216475 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:07.216436    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-89f7957c-gll94" event={"ID":"3ee001ad-dfbc-4571-ae40-9058341d8674","Type":"ContainerStarted","Data":"b0885bc60c08918eb82829ac2305ee7f7ead1629ca8da764c45f5a20624e2a00"}
Nov 04 03:50:08.225337 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:08.225301    5609 generic.go:356] "Generic (PLEG): container finished" podID="3ee001ad-dfbc-4571-ae40-9058341d8674" containerID="ef8273d5ad448d16086119c3ee9089de38c5d832703bf75137c77765c4dd6702" exitCode=0
Nov 04 03:50:08.225734 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:08.225384    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-89f7957c-gll94" event={"ID":"3ee001ad-dfbc-4571-ae40-9058341d8674","Type":"ContainerDied","Data":"ef8273d5ad448d16086119c3ee9089de38c5d832703bf75137c77765c4dd6702"}
Nov 04 03:50:09.364959 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:09.364936    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-89f7957c-gll94"
Nov 04 03:50:09.506251 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:09.506169    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3ee001ad-dfbc-4571-ae40-9058341d8674-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3ee001ad-dfbc-4571-ae40-9058341d8674-maas-api-xks-ca\") pod \"3ee001ad-dfbc-4571-ae40-9058341d8674\" (UID: \"3ee001ad-dfbc-4571-ae40-9058341d8674\") "
Nov 04 03:50:09.506434 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:09.506319    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3ee001ad-dfbc-4571-ae40-9058341d8674-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3ee001ad-dfbc-4571-ae40-9058341d8674-maas-api-openshift-ca\") pod \"3ee001ad-dfbc-4571-ae40-9058341d8674\" (UID: \"3ee001ad-dfbc-4571-ae40-9058341d8674\") "
Nov 04 03:50:09.506434 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:09.506395    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3ee001ad-dfbc-4571-ae40-9058341d8674-kube-api-access-bm948\" (UniqueName: \"kubernetes.io/projected/3ee001ad-dfbc-4571-ae40-9058341d8674-kube-api-access-bm948\") pod \"3ee001ad-dfbc-4571-ae40-9058341d8674\" (UID: \"3ee001ad-dfbc-4571-ae40-9058341d8674\") "
Nov 04 03:50:09.506713 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:09.506681    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ee001ad-dfbc-4571-ae40-9058341d8674-maas-api-openshift-ca" pod "3ee001ad-dfbc-4571-ae40-9058341d8674" (UID: "3ee001ad-dfbc-4571-ae40-9058341d8674"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:50:09.508506 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:09.508484    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ee001ad-dfbc-4571-ae40-9058341d8674-maas-api-xks-ca" pod "3ee001ad-dfbc-4571-ae40-9058341d8674" (UID: "3ee001ad-dfbc-4571-ae40-9058341d8674"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:50:09.508619 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:09.508581    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ee001ad-dfbc-4571-ae40-9058341d8674-kube-api-access-bm948" pod "3ee001ad-dfbc-4571-ae40-9058341d8674" (UID: "3ee001ad-dfbc-4571-ae40-9058341d8674"). InnerVolumeSpecName "kube-api-access-bm948". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:50:09.607595 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:09.607564    5609 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3ee001ad-dfbc-4571-ae40-9058341d8674-maas-api-openshift-ca\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:09.607735 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:09.607714    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bm948\" (UniqueName: \"kubernetes.io/projected/3ee001ad-dfbc-4571-ae40-9058341d8674-kube-api-access-bm948\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:09.607789 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:09.607743    5609 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3ee001ad-dfbc-4571-ae40-9058341d8674-maas-api-xks-ca\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:10.239574 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:10.239541    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-89f7957c-gll94"
Nov 04 03:50:10.239754 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:10.239566    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-89f7957c-gll94" event={"ID":"3ee001ad-dfbc-4571-ae40-9058341d8674","Type":"ContainerDied","Data":"b0885bc60c08918eb82829ac2305ee7f7ead1629ca8da764c45f5a20624e2a00"}
Nov 04 03:50:10.239754 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:10.239621    5609 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b0885bc60c08918eb82829ac2305ee7f7ead1629ca8da764c45f5a20624e2a00"
Nov 04 03:50:18.594065 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:50:18.594025    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:50:38.239710 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:38.239675    5609 scope.go:122] "RemoveContainer" containerID="222acfb5920b423acbaa02b44f6734ccd0b2ee9f58de6f73bc849aa69b1c2d7f"
Nov 04 03:50:38.248837 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:38.248809    5609 scope.go:122] "RemoveContainer" containerID="e8c47d823fc13018c1f775f9c1392f80c2c23ff4610531fb65335d30c05d4790"
Nov 04 03:50:40.541130 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:40.541102    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp"]
Nov 04 03:50:40.541610 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:40.541285    5609 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp" podUID="477d4138-273f-4c56-b997-d81f053d4f34" containerName="istio-proxy" containerID="cri-o://7d204598c80f1adf2f988bf8ca8d258be052856ca62149e7ec42a238e7bac9c4" gracePeriod=30
Nov 04 03:50:42.468268 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:42.468233    5609 patch_prober.go:29] interesting pod/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 03:50:42.468695 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:42.468298    5609 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp" podUID="477d4138-273f-4c56-b997-d81f053d4f34" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 03:50:45.796925 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:45.796902    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp"
Nov 04 03:50:45.866958 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:45.866894    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/477d4138-273f-4c56-b997-d81f053d4f34-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/477d4138-273f-4c56-b997-d81f053d4f34-istio-podinfo\") pod \"477d4138-273f-4c56-b997-d81f053d4f34\" (UID: \"477d4138-273f-4c56-b997-d81f053d4f34\") "
Nov 04 03:50:45.866958 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:45.866951    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/477d4138-273f-4c56-b997-d81f053d4f34-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/477d4138-273f-4c56-b997-d81f053d4f34-workload-socket\") pod \"477d4138-273f-4c56-b997-d81f053d4f34\" (UID: \"477d4138-273f-4c56-b997-d81f053d4f34\") "
Nov 04 03:50:45.867162 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:45.866986    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/477d4138-273f-4c56-b997-d81f053d4f34-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/477d4138-273f-4c56-b997-d81f053d4f34-istiod-ca-cert\") pod \"477d4138-273f-4c56-b997-d81f053d4f34\" (UID: \"477d4138-273f-4c56-b997-d81f053d4f34\") "
Nov 04 03:50:45.867162 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:45.867020    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/477d4138-273f-4c56-b997-d81f053d4f34-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/477d4138-273f-4c56-b997-d81f053d4f34-credential-socket\") pod \"477d4138-273f-4c56-b997-d81f053d4f34\" (UID: \"477d4138-273f-4c56-b997-d81f053d4f34\") "
Nov 04 03:50:45.867162 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:45.867037    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/477d4138-273f-4c56-b997-d81f053d4f34-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/477d4138-273f-4c56-b997-d81f053d4f34-istio-data\") pod \"477d4138-273f-4c56-b997-d81f053d4f34\" (UID: \"477d4138-273f-4c56-b997-d81f053d4f34\") "
Nov 04 03:50:45.867162 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:45.867100    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/477d4138-273f-4c56-b997-d81f053d4f34-istio-token\" (UniqueName: \"kubernetes.io/projected/477d4138-273f-4c56-b997-d81f053d4f34-istio-token\") pod \"477d4138-273f-4c56-b997-d81f053d4f34\" (UID: \"477d4138-273f-4c56-b997-d81f053d4f34\") "
Nov 04 03:50:45.867162 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:45.867128    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/477d4138-273f-4c56-b997-d81f053d4f34-kube-api-access-xjhc6\" (UniqueName: \"kubernetes.io/projected/477d4138-273f-4c56-b997-d81f053d4f34-kube-api-access-xjhc6\") pod \"477d4138-273f-4c56-b997-d81f053d4f34\" (UID: \"477d4138-273f-4c56-b997-d81f053d4f34\") "
Nov 04 03:50:45.867455 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:45.867174    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/477d4138-273f-4c56-b997-d81f053d4f34-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/477d4138-273f-4c56-b997-d81f053d4f34-istio-envoy\") pod \"477d4138-273f-4c56-b997-d81f053d4f34\" (UID: \"477d4138-273f-4c56-b997-d81f053d4f34\") "
Nov 04 03:50:45.867455 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:45.867206    5609 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/477d4138-273f-4c56-b997-d81f053d4f34-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/477d4138-273f-4c56-b997-d81f053d4f34-workload-certs\") pod \"477d4138-273f-4c56-b997-d81f053d4f34\" (UID: \"477d4138-273f-4c56-b997-d81f053d4f34\") "
Nov 04 03:50:45.867455 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:45.867273    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/477d4138-273f-4c56-b997-d81f053d4f34-workload-socket" pod "477d4138-273f-4c56-b997-d81f053d4f34" (UID: "477d4138-273f-4c56-b997-d81f053d4f34"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:50:45.867455 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:45.867334    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/477d4138-273f-4c56-b997-d81f053d4f34-credential-socket" pod "477d4138-273f-4c56-b997-d81f053d4f34" (UID: "477d4138-273f-4c56-b997-d81f053d4f34"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:50:45.867455 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:45.867345    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/477d4138-273f-4c56-b997-d81f053d4f34-istio-data" pod "477d4138-273f-4c56-b997-d81f053d4f34" (UID: "477d4138-273f-4c56-b997-d81f053d4f34"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:50:45.867717 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:45.867468    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/477d4138-273f-4c56-b997-d81f053d4f34-istiod-ca-cert" pod "477d4138-273f-4c56-b997-d81f053d4f34" (UID: "477d4138-273f-4c56-b997-d81f053d4f34"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:50:45.867717 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:45.867591    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/477d4138-273f-4c56-b997-d81f053d4f34-workload-certs" pod "477d4138-273f-4c56-b997-d81f053d4f34" (UID: "477d4138-273f-4c56-b997-d81f053d4f34"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:50:45.867717 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:45.867645    5609 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/477d4138-273f-4c56-b997-d81f053d4f34-workload-socket\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:45.867717 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:45.867658    5609 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/477d4138-273f-4c56-b997-d81f053d4f34-istiod-ca-cert\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:45.867717 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:45.867667    5609 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/477d4138-273f-4c56-b997-d81f053d4f34-credential-socket\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:45.867717 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:45.867675    5609 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/477d4138-273f-4c56-b997-d81f053d4f34-istio-data\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:45.869476 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:45.869447    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/477d4138-273f-4c56-b997-d81f053d4f34-kube-api-access-xjhc6" pod "477d4138-273f-4c56-b997-d81f053d4f34" (UID: "477d4138-273f-4c56-b997-d81f053d4f34"). InnerVolumeSpecName "kube-api-access-xjhc6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:50:45.869476 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:45.869466    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/477d4138-273f-4c56-b997-d81f053d4f34-istio-envoy" pod "477d4138-273f-4c56-b997-d81f053d4f34" (UID: "477d4138-273f-4c56-b997-d81f053d4f34"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:50:45.869660 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:45.869600    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/477d4138-273f-4c56-b997-d81f053d4f34-istio-podinfo" pod "477d4138-273f-4c56-b997-d81f053d4f34" (UID: "477d4138-273f-4c56-b997-d81f053d4f34"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:50:45.869795 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:45.869770    5609 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/477d4138-273f-4c56-b997-d81f053d4f34-istio-token" pod "477d4138-273f-4c56-b997-d81f053d4f34" (UID: "477d4138-273f-4c56-b997-d81f053d4f34"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:50:45.968558 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:45.968531    5609 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/477d4138-273f-4c56-b997-d81f053d4f34-istio-envoy\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:45.968558 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:45.968555    5609 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/477d4138-273f-4c56-b997-d81f053d4f34-workload-certs\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:45.968558 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:45.968564    5609 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/477d4138-273f-4c56-b997-d81f053d4f34-istio-podinfo\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:45.968820 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:45.968573    5609 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/477d4138-273f-4c56-b997-d81f053d4f34-istio-token\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:45.968820 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:45.968582    5609 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xjhc6\" (UniqueName: \"kubernetes.io/projected/477d4138-273f-4c56-b997-d81f053d4f34-kube-api-access-xjhc6\") on node \"ip-10-0-128-50.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:46.462193 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:46.462162    5609 generic.go:356] "Generic (PLEG): container finished" podID="477d4138-273f-4c56-b997-d81f053d4f34" containerID="7d204598c80f1adf2f988bf8ca8d258be052856ca62149e7ec42a238e7bac9c4" exitCode=0
Nov 04 03:50:46.462391 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:46.462212    5609 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp"
Nov 04 03:50:46.462391 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:46.462211    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp" event={"ID":"477d4138-273f-4c56-b997-d81f053d4f34","Type":"ContainerDied","Data":"7d204598c80f1adf2f988bf8ca8d258be052856ca62149e7ec42a238e7bac9c4"}
Nov 04 03:50:46.462391 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:46.462327    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp" event={"ID":"477d4138-273f-4c56-b997-d81f053d4f34","Type":"ContainerDied","Data":"4b7bbcc86aa0e8288552edefd7066e814eb9c72756a334a6b16dbceb05311600"}
Nov 04 03:50:46.462391 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:46.462349    5609 scope.go:122] "RemoveContainer" containerID="7d204598c80f1adf2f988bf8ca8d258be052856ca62149e7ec42a238e7bac9c4"
Nov 04 03:50:46.472265 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:46.472248    5609 scope.go:122] "RemoveContainer" containerID="fbb601ddfb0d9b3a25224652928d30b5f719e668dc1d4bbdc33994f18d7759e5"
Nov 04 03:50:46.480803 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:46.480785    5609 scope.go:122] "RemoveContainer" containerID="7d204598c80f1adf2f988bf8ca8d258be052856ca62149e7ec42a238e7bac9c4"
Nov 04 03:50:46.481059 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:50:46.481039    5609 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d204598c80f1adf2f988bf8ca8d258be052856ca62149e7ec42a238e7bac9c4\": container with ID starting with 7d204598c80f1adf2f988bf8ca8d258be052856ca62149e7ec42a238e7bac9c4 not found: ID does not exist" containerID="7d204598c80f1adf2f988bf8ca8d258be052856ca62149e7ec42a238e7bac9c4"
Nov 04 03:50:46.481128 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:46.481072    5609 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d204598c80f1adf2f988bf8ca8d258be052856ca62149e7ec42a238e7bac9c4"} err="failed to get container status \"7d204598c80f1adf2f988bf8ca8d258be052856ca62149e7ec42a238e7bac9c4\": rpc error: code = NotFound desc = could not find container \"7d204598c80f1adf2f988bf8ca8d258be052856ca62149e7ec42a238e7bac9c4\": container with ID starting with 7d204598c80f1adf2f988bf8ca8d258be052856ca62149e7ec42a238e7bac9c4 not found: ID does not exist"
Nov 04 03:50:46.481128 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:46.481096    5609 scope.go:122] "RemoveContainer" containerID="fbb601ddfb0d9b3a25224652928d30b5f719e668dc1d4bbdc33994f18d7759e5"
Nov 04 03:50:46.481328 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:50:46.481311    5609 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbb601ddfb0d9b3a25224652928d30b5f719e668dc1d4bbdc33994f18d7759e5\": container with ID starting with fbb601ddfb0d9b3a25224652928d30b5f719e668dc1d4bbdc33994f18d7759e5 not found: ID does not exist" containerID="fbb601ddfb0d9b3a25224652928d30b5f719e668dc1d4bbdc33994f18d7759e5"
Nov 04 03:50:46.481370 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:46.481333    5609 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbb601ddfb0d9b3a25224652928d30b5f719e668dc1d4bbdc33994f18d7759e5"} err="failed to get container status \"fbb601ddfb0d9b3a25224652928d30b5f719e668dc1d4bbdc33994f18d7759e5\": rpc error: code = NotFound desc = could not find container \"fbb601ddfb0d9b3a25224652928d30b5f719e668dc1d4bbdc33994f18d7759e5\": container with ID starting with fbb601ddfb0d9b3a25224652928d30b5f719e668dc1d4bbdc33994f18d7759e5 not found: ID does not exist"
Nov 04 03:50:46.621293 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:46.621267    5609 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp"]
Nov 04 03:50:46.668370 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:46.668336    5609 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-tg2bp"]
Nov 04 03:50:47.535079 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:47.535043    5609 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="477d4138-273f-4c56-b997-d81f053d4f34" path="/var/lib/kubelet/pods/477d4138-273f-4c56-b997-d81f053d4f34/volumes"
Nov 04 03:50:48.559494 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:50:48.559463    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:50:51.677281 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:51.677244    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5cb7d797b8-rjd4x_b09e35e7-3a7f-4350-a3ef-61ba60f041a2/authorino/0.log"
Nov 04 03:50:52.089854 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:52.089768    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-hshfp_894e9896-b132-40b9-9fb2-8a5c7f60703c/manager/0.log"
Nov 04 03:50:52.332258 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:52.332228    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-zttlw_6923e755-81bb-447e-923a-ff5d745f8a84/manager/0.log"
Nov 04 03:50:55.480083 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:50:55.480045    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-4hmfk_923229c3-ab54-46ca-ac3a-fd9e951116c7/manager/0.log"
Nov 04 03:51:08.006259 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:51:08.006229    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-qhqnk_bb2a6c09-2ea4-407c-985f-5dfa99b191e3/manager/0.log"
Nov 04 03:51:08.926327 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:51:08.926294    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5fb896bc89-fkk4g_e6af8c02-22bc-49e1-adb9-2f01da0af47c/manager/1.log"
Nov 04 03:51:10.027384 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:51:10.027353    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-98hr6_c8183d45-a9c7-4fe4-bbfa-a1ddc7e33423/proxy/0.log"
Nov 04 03:51:10.702695 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:51:10.702663    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5fb896bc89-fkk4g_e6af8c02-22bc-49e1-adb9-2f01da0af47c/manager/0.log"
Nov 04 03:51:14.507668 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:51:14.507630    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5cb7d797b8-rjd4x_b09e35e7-3a7f-4350-a3ef-61ba60f041a2/authorino/0.log"
Nov 04 03:51:18.558848 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:51:18.558812    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:51:46.873488 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:51:46.873457    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-qhqnk_bb2a6c09-2ea4-407c-985f-5dfa99b191e3/manager/0.log"
Nov 04 03:51:47.741022 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:51:47.740980    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5fb896bc89-fkk4g_e6af8c02-22bc-49e1-adb9-2f01da0af47c/manager/1.log"
Nov 04 03:51:48.558974 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:51:48.558939    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:51:48.782128 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:51:48.782101    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-98hr6_c8183d45-a9c7-4fe4-bbfa-a1ddc7e33423/proxy/0.log"
Nov 04 03:51:49.450990 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:51:49.450956    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5fb896bc89-fkk4g_e6af8c02-22bc-49e1-adb9-2f01da0af47c/manager/0.log"
Nov 04 03:51:50.485146 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:51:50.485120    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-0973bc47-z9rdn_0ae0eefa-fbd0-4073-ab76-a291aedd86e1/revoke-keys/0.log"
Nov 04 03:51:50.708817 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:51:50.708778    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-89f7957c-gll94_3ee001ad-dfbc-4571-ae40-9058341d8674/revoke-keys/0.log"
Nov 04 03:51:50.930428 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:51:50.930381    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-f020be60-4zcsf_47926061-d052-49d9-bc64-73b38cd951d5/revoke-keys/0.log"
Nov 04 03:51:51.169185 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:51:51.169153    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-mp4fd_4db3855d-8b8d-4b57-a83f-1a8da42cda50/postgres/0.log"
Nov 04 03:51:53.142761 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:51:53.142731    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5cb7d797b8-rjd4x_b09e35e7-3a7f-4350-a3ef-61ba60f041a2/authorino/0.log"
Nov 04 03:51:53.584076 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:51:53.584045    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-hshfp_894e9896-b132-40b9-9fb2-8a5c7f60703c/manager/0.log"
Nov 04 03:51:53.822758 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:51:53.822720    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-zttlw_6923e755-81bb-447e-923a-ff5d745f8a84/manager/0.log"
Nov 04 03:51:54.920561 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:51:54.920518    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-4hmfk_923229c3-ab54-46ca-ac3a-fd9e951116c7/manager/0.log"
Nov 04 03:51:56.541817 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:51:56.541780    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c_416ff65b-51ec-42e2-a4da-071eb613eac6/istio-proxy/0.log"
Nov 04 03:51:56.797229 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:51:56.797149    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-pmscg_22e952c0-bbe0-4c13-ab52-31e9a96aab29/discovery/0.log"
Nov 04 03:51:57.517670 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:51:57.517636    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8b64f9ff9-26xvk_621375f0-75dd-445b-8e50-50b52600a7dc/istio-proxy/0.log"
Nov 04 03:51:58.535249 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:51:58.535220    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8b64f9ff9-hgwbk_fd1ae838-b620-411d-bd66-82c7d32cc41a/istio-proxy/0.log"
Nov 04 03:51:59.728245 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:51:59.728214    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8b64f9ff9-v5h77_8772b2ff-e4cc-4b41-85a9-491fbaf001c8/istio-proxy/1.log"
Nov 04 03:52:00.071775 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:00.071702    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8_fc5b4561-6c05-4d20-8483-e4c83f3fef52/istio-proxy/0.log"
Nov 04 03:52:00.771959 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:00.771925    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7cfb798487-kbfzk_c1b21c41-4b51-4da8-8920-c38cf49554bf/router/0.log"
Nov 04 03:52:01.508216 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:01.508182    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8b64f9ff9-v5h77_8772b2ff-e4cc-4b41-85a9-491fbaf001c8/istio-proxy/0.log"
Nov 04 03:52:03.262450 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:03.262421    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-tbfbx_a3ab7fb6-0416-4f11-85f4-555ffc5e0c56/main/0.log"
Nov 04 03:52:13.561593 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:13.561559    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ldhc4/must-gather-8qdxb"]
Nov 04 03:52:13.561988 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:13.561949    5609 cpu_manager.go:414] "containerMap: removing container" podUID="477d4138-273f-4c56-b997-d81f053d4f34" containerName="istio-proxy"
Nov 04 03:52:13.561988 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:13.561959    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="477d4138-273f-4c56-b997-d81f053d4f34" containerName="istio-proxy"
Nov 04 03:52:13.561988 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:13.561970    5609 cpu_manager.go:414] "containerMap: removing container" podUID="3ee001ad-dfbc-4571-ae40-9058341d8674" containerName="revoke-keys"
Nov 04 03:52:13.561988 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:13.561974    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3ee001ad-dfbc-4571-ae40-9058341d8674" containerName="revoke-keys"
Nov 04 03:52:13.561988 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:13.561986    5609 cpu_manager.go:414] "containerMap: removing container" podUID="477d4138-273f-4c56-b997-d81f053d4f34" containerName="istio-proxy"
Nov 04 03:52:13.561988 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:13.561991    5609 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="477d4138-273f-4c56-b997-d81f053d4f34" containerName="istio-proxy"
Nov 04 03:52:13.562169 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:13.562069    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="477d4138-273f-4c56-b997-d81f053d4f34" containerName="istio-proxy"
Nov 04 03:52:13.562169 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:13.562076    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="3ee001ad-dfbc-4571-ae40-9058341d8674" containerName="revoke-keys"
Nov 04 03:52:13.562226 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:13.562206    5609 memory_manager.go:365] "RemoveStaleState removing state" podUID="477d4138-273f-4c56-b997-d81f053d4f34" containerName="istio-proxy"
Nov 04 03:52:13.565731 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:13.565711    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ldhc4/must-gather-8qdxb"
Nov 04 03:52:13.578116 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:13.578089    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d6409c70-4174-4913-aa2f-59988a110899-must-gather-output\") pod \"must-gather-8qdxb\" (UID: \"d6409c70-4174-4913-aa2f-59988a110899\") " pod="openshift-must-gather-ldhc4/must-gather-8qdxb"
Nov 04 03:52:13.578244 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:13.578126    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hx5s2\" (UniqueName: \"kubernetes.io/projected/d6409c70-4174-4913-aa2f-59988a110899-kube-api-access-hx5s2\") pod \"must-gather-8qdxb\" (UID: \"d6409c70-4174-4913-aa2f-59988a110899\") " pod="openshift-must-gather-ldhc4/must-gather-8qdxb"
Nov 04 03:52:13.663874 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:13.663839    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-ldhc4\"/\"default-dockercfg-8mqql\""
Nov 04 03:52:13.663874 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:13.663854    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ldhc4\"/\"kube-root-ca.crt\""
Nov 04 03:52:13.663874 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:13.663854    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ldhc4\"/\"openshift-service-ca.crt\""
Nov 04 03:52:13.666493 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:13.666464    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ldhc4/must-gather-8qdxb"]
Nov 04 03:52:13.679302 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:13.679279    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx5s2\" (UniqueName: \"kubernetes.io/projected/d6409c70-4174-4913-aa2f-59988a110899-kube-api-access-hx5s2\") pod \"must-gather-8qdxb\" (UID: \"d6409c70-4174-4913-aa2f-59988a110899\") " pod="openshift-must-gather-ldhc4/must-gather-8qdxb"
Nov 04 03:52:13.679486 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:13.679469    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d6409c70-4174-4913-aa2f-59988a110899-must-gather-output\") pod \"must-gather-8qdxb\" (UID: \"d6409c70-4174-4913-aa2f-59988a110899\") " pod="openshift-must-gather-ldhc4/must-gather-8qdxb"
Nov 04 03:52:13.679803 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:13.679784    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d6409c70-4174-4913-aa2f-59988a110899-must-gather-output\") pod \"must-gather-8qdxb\" (UID: \"d6409c70-4174-4913-aa2f-59988a110899\") " pod="openshift-must-gather-ldhc4/must-gather-8qdxb"
Nov 04 03:52:13.738202 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:13.738175    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hx5s2\" (UniqueName: \"kubernetes.io/projected/d6409c70-4174-4913-aa2f-59988a110899-kube-api-access-hx5s2\") pod \"must-gather-8qdxb\" (UID: \"d6409c70-4174-4913-aa2f-59988a110899\") " pod="openshift-must-gather-ldhc4/must-gather-8qdxb"
Nov 04 03:52:13.880130 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:13.880045    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ldhc4/must-gather-8qdxb"
Nov 04 03:52:14.254877 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:14.254843    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ldhc4/must-gather-8qdxb"]
Nov 04 03:52:14.260212 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:52:14.260184    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd6409c70_4174_4913_aa2f_59988a110899.slice/crio-b0158bc45d4c62ecf08f24fca03ae15ebe70fca73fd6f8d5579d66b5f9ad0276 WatchSource:0}: Error finding container b0158bc45d4c62ecf08f24fca03ae15ebe70fca73fd6f8d5579d66b5f9ad0276: Status 404 returned error can't find the container with id b0158bc45d4c62ecf08f24fca03ae15ebe70fca73fd6f8d5579d66b5f9ad0276
Nov 04 03:52:14.262257 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:14.262240    5609 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 03:52:14.983880 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:14.983845    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ldhc4/must-gather-8qdxb" event={"ID":"d6409c70-4174-4913-aa2f-59988a110899","Type":"ContainerStarted","Data":"b0158bc45d4c62ecf08f24fca03ae15ebe70fca73fd6f8d5579d66b5f9ad0276"}
Nov 04 03:52:18.567084 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:52:18.566996    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:52:22.039228 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:22.039182    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ldhc4/must-gather-8qdxb" event={"ID":"d6409c70-4174-4913-aa2f-59988a110899","Type":"ContainerStarted","Data":"04c429b2d3f6ec76c035bdc2eed7cff095587a4654acca7a60193bc4ceb2cee2"}
Nov 04 03:52:22.039705 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:22.039239    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ldhc4/must-gather-8qdxb" event={"ID":"d6409c70-4174-4913-aa2f-59988a110899","Type":"ContainerStarted","Data":"c3031cbef74ff15f3148c51abe88d96eecd83c9c7d544f26449a49e0542a5281"}
Nov 04 03:52:22.150454 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:22.150354    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-ldhc4/must-gather-8qdxb" podStartSLOduration=2.450415294 podStartE2EDuration="9.150337345s" totalImagesPullingTime="6.699922051s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:52:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:52:22.149321242 +0000 UTC m=+2084.725974158" watchObservedRunningTime="2026-11-04 03:52:22.150337345 +0000 UTC m=+2084.726990260"
Nov 04 03:52:24.171931 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:24.171901    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-tkkhk_d10e9268-df07-428b-ae25-528d9fe075d2/global-pull-secret-syncer/0.log"
Nov 04 03:52:25.471069 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:25.471042    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-mdchx_e0c72601-2624-4632-9f15-c19fcc489aae/connection-checker/0.log"
Nov 04 03:52:25.622755 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:25.622726    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-qhqnk_bb2a6c09-2ea4-407c-985f-5dfa99b191e3/manager/0.log"
Nov 04 03:52:26.009589 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:26.009561    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-n6r7s_10cd06d4-ef1a-4ecf-a72c-e6d0084f3032/connection-checker/0.log"
Nov 04 03:52:26.288853 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:26.288758    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5fb896bc89-fkk4g_e6af8c02-22bc-49e1-adb9-2f01da0af47c/manager/0.log"
Nov 04 03:52:26.380668 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:26.380635    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5fb896bc89-fkk4g_e6af8c02-22bc-49e1-adb9-2f01da0af47c/manager/1.log"
Nov 04 03:52:26.552947 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:26.552876    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-xpmwt_a51f26a7-0ec8-4c56-881a-98c6d625478b/connection-checker/0.log"
Nov 04 03:52:27.495836 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:27.495807    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-m2vkr_029e7b7b-265e-459a-bc09-a90b5c179993/konnectivity-agent/0.log"
Nov 04 03:52:27.498177 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:27.498156    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-98hr6_c8183d45-a9c7-4fe4-bbfa-a1ddc7e33423/proxy/0.log"
Nov 04 03:52:28.366217 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:28.366170    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-50.us-west-2.compute.internal_aeb0f7512125881a2a5e1813eb42a8f7/haproxy/0.log"
Nov 04 03:52:30.926611 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:30.926576    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-qhqnk_bb2a6c09-2ea4-407c-985f-5dfa99b191e3/manager/0.log"
Nov 04 03:52:31.455438 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:31.455408    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5fb896bc89-fkk4g_e6af8c02-22bc-49e1-adb9-2f01da0af47c/manager/0.log"
Nov 04 03:52:31.544713 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:31.544680    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5fb896bc89-fkk4g_e6af8c02-22bc-49e1-adb9-2f01da0af47c/manager/1.log"
Nov 04 03:52:32.693698 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:32.693667    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-98hr6_c8183d45-a9c7-4fe4-bbfa-a1ddc7e33423/proxy/0.log"
Nov 04 03:52:33.405909 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:33.405874    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5cb7d797b8-rjd4x_b09e35e7-3a7f-4350-a3ef-61ba60f041a2/authorino/0.log"
Nov 04 03:52:33.716951 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:33.716915    5609 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-x26lf/perf-node-gather-daemonset-xmx97"]
Nov 04 03:52:33.721526 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:33.721496    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-xmx97"
Nov 04 03:52:33.757750 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:33.757725    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-hshfp_894e9896-b132-40b9-9fb2-8a5c7f60703c/manager/0.log"
Nov 04 03:52:33.770332 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:33.770307    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2f68b813-e0a0-4a7f-bb6a-f184d5b70790-sys\") pod \"perf-node-gather-daemonset-xmx97\" (UID: \"2f68b813-e0a0-4a7f-bb6a-f184d5b70790\") " pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-xmx97"
Nov 04 03:52:33.770470 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:33.770357    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2f68b813-e0a0-4a7f-bb6a-f184d5b70790-lib-modules\") pod \"perf-node-gather-daemonset-xmx97\" (UID: \"2f68b813-e0a0-4a7f-bb6a-f184d5b70790\") " pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-xmx97"
Nov 04 03:52:33.770470 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:33.770459    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnp4m\" (UniqueName: \"kubernetes.io/projected/2f68b813-e0a0-4a7f-bb6a-f184d5b70790-kube-api-access-pnp4m\") pod \"perf-node-gather-daemonset-xmx97\" (UID: \"2f68b813-e0a0-4a7f-bb6a-f184d5b70790\") " pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-xmx97"
Nov 04 03:52:33.770544 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:33.770491    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2f68b813-e0a0-4a7f-bb6a-f184d5b70790-proc\") pod \"perf-node-gather-daemonset-xmx97\" (UID: \"2f68b813-e0a0-4a7f-bb6a-f184d5b70790\") " pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-xmx97"
Nov 04 03:52:33.770614 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:33.770596    5609 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2f68b813-e0a0-4a7f-bb6a-f184d5b70790-podres\") pod \"perf-node-gather-daemonset-xmx97\" (UID: \"2f68b813-e0a0-4a7f-bb6a-f184d5b70790\") " pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-xmx97"
Nov 04 03:52:33.785573 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:33.785550    5609 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-x26lf\"/\"default-dockercfg-l6db9\""
Nov 04 03:52:33.821341 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:33.821318    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-x26lf/perf-node-gather-daemonset-xmx97"]
Nov 04 03:52:33.860328 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:33.860303    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-x26lf\"/\"openshift-service-ca.crt\""
Nov 04 03:52:33.860471 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:33.860421    5609 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-x26lf\"/\"kube-root-ca.crt\""
Nov 04 03:52:33.871577 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:33.871552    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2f68b813-e0a0-4a7f-bb6a-f184d5b70790-sys\") pod \"perf-node-gather-daemonset-xmx97\" (UID: \"2f68b813-e0a0-4a7f-bb6a-f184d5b70790\") " pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-xmx97"
Nov 04 03:52:33.871661 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:33.871602    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2f68b813-e0a0-4a7f-bb6a-f184d5b70790-lib-modules\") pod \"perf-node-gather-daemonset-xmx97\" (UID: \"2f68b813-e0a0-4a7f-bb6a-f184d5b70790\") " pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-xmx97"
Nov 04 03:52:33.871699 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:33.871690    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2f68b813-e0a0-4a7f-bb6a-f184d5b70790-lib-modules\") pod \"perf-node-gather-daemonset-xmx97\" (UID: \"2f68b813-e0a0-4a7f-bb6a-f184d5b70790\") " pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-xmx97"
Nov 04 03:52:33.871745 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:33.871730    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2f68b813-e0a0-4a7f-bb6a-f184d5b70790-sys\") pod \"perf-node-gather-daemonset-xmx97\" (UID: \"2f68b813-e0a0-4a7f-bb6a-f184d5b70790\") " pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-xmx97"
Nov 04 03:52:33.871789 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:33.871757    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnp4m\" (UniqueName: \"kubernetes.io/projected/2f68b813-e0a0-4a7f-bb6a-f184d5b70790-kube-api-access-pnp4m\") pod \"perf-node-gather-daemonset-xmx97\" (UID: \"2f68b813-e0a0-4a7f-bb6a-f184d5b70790\") " pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-xmx97"
Nov 04 03:52:33.871789 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:33.871779    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2f68b813-e0a0-4a7f-bb6a-f184d5b70790-proc\") pod \"perf-node-gather-daemonset-xmx97\" (UID: \"2f68b813-e0a0-4a7f-bb6a-f184d5b70790\") " pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-xmx97"
Nov 04 03:52:33.871859 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:33.871836    5609 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2f68b813-e0a0-4a7f-bb6a-f184d5b70790-podres\") pod \"perf-node-gather-daemonset-xmx97\" (UID: \"2f68b813-e0a0-4a7f-bb6a-f184d5b70790\") " pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-xmx97"
Nov 04 03:52:33.871919 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:33.871903    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2f68b813-e0a0-4a7f-bb6a-f184d5b70790-proc\") pod \"perf-node-gather-daemonset-xmx97\" (UID: \"2f68b813-e0a0-4a7f-bb6a-f184d5b70790\") " pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-xmx97"
Nov 04 03:52:33.871954 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:33.871941    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2f68b813-e0a0-4a7f-bb6a-f184d5b70790-podres\") pod \"perf-node-gather-daemonset-xmx97\" (UID: \"2f68b813-e0a0-4a7f-bb6a-f184d5b70790\") " pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-xmx97"
Nov 04 03:52:33.931828 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:33.931803    5609 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnp4m\" (UniqueName: \"kubernetes.io/projected/2f68b813-e0a0-4a7f-bb6a-f184d5b70790-kube-api-access-pnp4m\") pod \"perf-node-gather-daemonset-xmx97\" (UID: \"2f68b813-e0a0-4a7f-bb6a-f184d5b70790\") " pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-xmx97"
Nov 04 03:52:34.032954 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:34.032868    5609 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-xmx97"
Nov 04 03:52:34.217028 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:34.216994    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-zttlw_6923e755-81bb-447e-923a-ff5d745f8a84/manager/0.log"
Nov 04 03:52:34.408415 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:34.408340    5609 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-x26lf/perf-node-gather-daemonset-xmx97"]
Nov 04 03:52:34.413626 ip-10-0-128-50 kubenswrapper[5609]: W1104 03:52:34.413586    5609 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2f68b813_e0a0_4a7f_bb6a_f184d5b70790.slice/crio-dbb53f6d62aa4c752758e842ba78684c2d0bef900555b262f1fb37bbab8c8e82 WatchSource:0}: Error finding container dbb53f6d62aa4c752758e842ba78684c2d0bef900555b262f1fb37bbab8c8e82: Status 404 returned error can't find the container with id dbb53f6d62aa4c752758e842ba78684c2d0bef900555b262f1fb37bbab8c8e82
Nov 04 03:52:35.121928 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:35.121890    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-xmx97" event={"ID":"2f68b813-e0a0-4a7f-bb6a-f184d5b70790","Type":"ContainerStarted","Data":"0cd38770bf7adb869cd19ab981719285cb8f779a52baf54210411ada5c95b370"}
Nov 04 03:52:35.122365 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:35.121949    5609 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-xmx97" event={"ID":"2f68b813-e0a0-4a7f-bb6a-f184d5b70790","Type":"ContainerStarted","Data":"dbb53f6d62aa4c752758e842ba78684c2d0bef900555b262f1fb37bbab8c8e82"}
Nov 04 03:52:35.122365 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:35.121978    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-xmx97"
Nov 04 03:52:35.240383 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:35.240331    5609 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-xmx97" podStartSLOduration=2.240312843 podStartE2EDuration="2.240312843s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:52:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:52:35.236756978 +0000 UTC m=+2097.813409893" watchObservedRunningTime="2026-11-04 03:52:35.240312843 +0000 UTC m=+2097.816965759"
Nov 04 03:52:35.577498 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:35.577441    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-4hmfk_923229c3-ab54-46ca-ac3a-fd9e951116c7/manager/0.log"
Nov 04 03:52:36.230414 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:36.230357    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-qhqnk_bb2a6c09-2ea4-407c-985f-5dfa99b191e3/manager/0.log"
Nov 04 03:52:36.742743 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:36.742716    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5fb896bc89-fkk4g_e6af8c02-22bc-49e1-adb9-2f01da0af47c/manager/0.log"
Nov 04 03:52:36.839759 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:36.839731    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5fb896bc89-fkk4g_e6af8c02-22bc-49e1-adb9-2f01da0af47c/manager/1.log"
Nov 04 03:52:37.723710 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:37.723680    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-98hr6_c8183d45-a9c7-4fe4-bbfa-a1ddc7e33423/proxy/0.log"
Nov 04 03:52:37.956324 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:37.956299    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-p9989_068619f8-7fbb-4b22-ac7d-a1f144e084c0/console-operator/2.log"
Nov 04 03:52:37.963149 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:37.963130    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vmw5z_21a42913-5c4e-4057-aae5-2804038f16e4/ovn-acl-logging/0.log"
Nov 04 03:52:37.973317 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:37.973296    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-p9989_068619f8-7fbb-4b22-ac7d-a1f144e084c0/console-operator/2.log"
Nov 04 03:52:37.980832 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:37.980779    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vmw5z_21a42913-5c4e-4057-aae5-2804038f16e4/ovn-acl-logging/0.log"
Nov 04 03:52:41.096006 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:41.095976    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-67b5c995fcd6p9c_416ff65b-51ec-42e2-a4da-071eb613eac6/istio-proxy/0.log"
Nov 04 03:52:41.138778 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:41.138749    5609 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-xmx97"
Nov 04 03:52:42.346746 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:42.346714    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-884576c57-gmt9g_e87dded5-1e1b-45cb-ab2e-ae12550dfca7/cluster-monitoring-operator/0.log"
Nov 04 03:52:42.908928 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:42.908888    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-pmscg_22e952c0-bbe0-4c13-ab52-31e9a96aab29/discovery/0.log"
Nov 04 03:52:43.477700 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:43.477667    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8b64f9ff9-26xvk_621375f0-75dd-445b-8e50-50b52600a7dc/istio-proxy/0.log"
Nov 04 03:52:44.391935 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:44.391906    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8b64f9ff9-hgwbk_fd1ae838-b620-411d-bd66-82c7d32cc41a/istio-proxy/0.log"
Nov 04 03:52:45.520374 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:45.520340    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8b64f9ff9-v5h77_8772b2ff-e4cc-4b41-85a9-491fbaf001c8/istio-proxy/0.log"
Nov 04 03:52:45.522049 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:45.522032    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8b64f9ff9-v5h77_8772b2ff-e4cc-4b41-85a9-491fbaf001c8/istio-proxy/1.log"
Nov 04 03:52:45.643971 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:45.643913    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-77b95bc695-z75k8_fc5b4561-6c05-4d20-8483-e4c83f3fef52/istio-proxy/0.log"
Nov 04 03:52:46.294969 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:46.294930    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rmxkn_0866082f-d2ee-42e8-a599-f567af165fbe/node-exporter/0.log"
Nov 04 03:52:46.331508 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:46.331482    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7cfb798487-kbfzk_c1b21c41-4b51-4da8-8920-c38cf49554bf/router/0.log"
Nov 04 03:52:46.771734 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:46.771698    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rmxkn_0866082f-d2ee-42e8-a599-f567af165fbe/kube-rbac-proxy/0.log"
Nov 04 03:52:47.232748 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:47.232721    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rmxkn_0866082f-d2ee-42e8-a599-f567af165fbe/init-textfile/0.log"
Nov 04 03:52:48.559118 ip-10-0-128-50 kubenswrapper[5609]: E1104 03:52:48.559080    5609 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-46btb" containerName="extractor"
Nov 04 03:52:50.406924 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:50.406883    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5cb7d797b8-rjd4x_b09e35e7-3a7f-4350-a3ef-61ba60f041a2/authorino/0.log"
Nov 04 03:52:50.529901 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:50.529871    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-hshfp_894e9896-b132-40b9-9fb2-8a5c7f60703c/manager/0.log"
Nov 04 03:52:50.758313 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:50.758282    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-zttlw_6923e755-81bb-447e-923a-ff5d745f8a84/manager/0.log"
Nov 04 03:52:51.677667 ip-10-0-128-50 kubenswrapper[5609]: I1104 03:52:51.677628    5609 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-4hmfk_923229c3-ab54-46ca-ac3a-fd9e951116c7/manager/0.log"